From cbaa689a0feadfea83ca6ba5e2b1acada2ca1cf0 Mon Sep 17 00:00:00 2001 From: AAA <35992542+TNTcraftHIM@users.noreply.github.com> Date: Tue, 25 Aug 2026 16:29:07 +0800 Subject: [PATCH] docs(media): align phase truth (#259) --- .../0005-automatic-hybrid-media-routing.md | 30 +- ...07-path-isolated-representation-quality.md | 203 ++-- docs/deployment.md | 29 +- docs/project-memory.md | 157 ++- docs/research/browser-background-capture.md | 51 +- docs/research/browser-screen-audio-quality.md | 672 +++---------- docs/research/realtime-quality-adaptation.md | 937 +++--------------- docs/research/webrtc-p2p-screen-sharing.md | 2 +- docs/session-handoff.md | 88 +- docs/status.md | 89 +- docs/todo.md | 98 +- docs/verification-status.md | 134 ++- docs/方案设计.md | 24 +- docs/需求理解.md | 28 +- 14 files changed, 773 insertions(+), 1769 deletions(-) diff --git a/docs/adr/0005-automatic-hybrid-media-routing.md b/docs/adr/0005-automatic-hybrid-media-routing.md index 705070c..d1f0f6f 100644 --- a/docs/adr/0005-automatic-hybrid-media-routing.md +++ b/docs/adr/0005-automatic-hybrid-media-routing.md @@ -1,6 +1,6 @@ # ADR-0005: Automatic Hybrid Media Routing -- Status: The route model is accepted and deployed in exact release `1d87615` +- Status: The route model is accepted and deployed in exact release `af348ee` on strict v12, including Browser SFU ICE-server isolation, no product diagnostic-download UI, and the retained Host-only acceptance snapshot. - Date: 2026-08-20 @@ -271,6 +271,14 @@ second mutable graph. state sampler is separate from the pending candidate's short-lived first-frame observer and does not treat bitrate, FPS, track availability, or SFU layer choice as route authority. +- Browser page resume/visibility recovery rebaselines this same deadline before + another no-progress sample can invalidate an edge. Frozen JavaScript wall time + is not media-failure proof. A newly composited current-generation frame clears + stale local recovery presentation but does not create route authority. +- Manual media reconnect remains on the current exact route: a peer rebuild uses + the same parent, and an SFU reconnect reconstructs the same subscription. + Neither action performs quality selection or active reparenting. Only actual + recovery exhaustion reports route failure and enters normal reconciliation. - Authoritative pause aborts the pending child operation, including its current candidate and reservations, keeps the active graph, suppresses decoded-frame- stall decisions, and leaves new participants waiting. Resume always wakes a @@ -365,18 +373,14 @@ deployment-wide. ## Current Source And Deployment Boundary -Production runs exact deployed application/runtime revision -`1d8761528d0dba43fb6d818df3934483ba2f5340`, release `1d87615`; canonical `main` -contains newer source pending deployment, including the active-SFU cadence correction -and exact Browser SFU ICE-server-isolation implementation -`ae09c760adec76fd26da611d4928486d105c6d3b`. Both run strict `screener-v12` and -the direct/STUN peer plus LiveKit SFU/UDP route model. Browser SFU PCs use empty -external ICE-server lists. Chrome 151 on that exact production verified one -publisher and one subscriber advancing frames over LiveKit UDP; the controlled -exact-candidate rollback/SFU commit gate is also closed. The operation owner -remains only `route`. Exact evidence is owned by the -[verification ledger](../verification-status.md); broader heterogeneous-network -and SFU lifecycle validation remains open. +Production and canonical `main` run exact application/runtime revision +`af348ee1d508a3af02b18a7f46c461953798e19d`, release `af348ee`, on strict +`screener-v12` with the direct/STUN peer plus LiveKit SFU/UDP route model. +Browser SFU PCs use empty external ICE-server lists. The controlled exact- +candidate rollback/SFU commit and active-SFU cadence gates are closed. The +operation owner remains only `route`; broader heterogeneous-network and SFU +lifecycle validation remains open in the +[verification ledger](../verification-status.md). ## Acceptance Boundary diff --git a/docs/adr/0007-path-isolated-representation-quality.md b/docs/adr/0007-path-isolated-representation-quality.md index 340b87a..4c62d86 100644 --- a/docs/adr/0007-path-isolated-representation-quality.md +++ b/docs/adr/0007-path-isolated-representation-quality.md @@ -1,4 +1,4 @@ -# ADR-0007: LiveKit-Owned SFU Representation Adaptation +# ADR-0007: Framework-Owned Media Quality Adaptation - Status: Accepted, implemented, and deployed - Date: 2026-08-19 @@ -6,146 +6,101 @@ ## Context -Direct and peer paths already have independent `RTCPeerConnection` congestion -controllers. The SFU path is different: one Host publication serves several -subscribers, so a weak subscriber needs a lower encoded representation without -lowering every healthy subscriber or creating one encoder per Viewer. +Screener sends one realtime game-screen source through independent P2P +connections and, when needed, one shared LiveKit publication. WebRTC and +LiveKit already own congestion control, encoder adaptation, retransmission, +simulcast construction, subscriber bandwidth estimation, and layer selection. -A single non-scalable full-resolution representation cannot provide that spatial -downshift because an SFU forwards encoded packets and does not transcode them. -The earlier production A/B compared an always-active two-encoding publication -against its lower representation disabled. It showed that the always-active -lower representation competed with the full representation on that constrained -Host-to-SFU path; it did not test -LiveKit-owned demand-driven layer control and does not justify a permanent -single-encoding contract. +Earlier experiments added a Screener-defined lower SFU encoding and treated +quality preference as an application policy. Production evidence showed that +an always-active lower encoding can compete with the highest encoding on a +constrained Host-to-SFU path. It did not justify replacing LiveKit's native +adaptation with a single HIGH representation or another application controller. -The earlier application-layer controller discussion started from the false -premise that LiveKit/WebRTC did not already own per-subscriber bandwidth -estimation and layer selection. Built-in media control is the default owner. -Screener supplies the bounded representations and product ceilings but does not -reimplement that controller. A resource-constrained Host can explicitly choose -a lower existing share profile; the product does not protect it by silently -removing the representation required by constrained Viewers. +The product needs standard game-motion intent and user-selected ceilings, but +it does not need a second bitrate, resolution, FPS, or layer-control system. ## Decision -1. Direct and peer paths retain independent stock WebRTC congestion control. - Viewer feedback is never aggregated into a room-wide target. -2. The Browser Host SFU publication provides two bounded VP8 screen-share - representations: one half-resolution encoding and the original full-resolution - encoding. The lower representation keeps at most 30 fps and derives its - bitrate from the pinned SDK's pixel-and-frame-rate formula; the representation - count is fixed and does not grow with Viewer count. -3. LiveKit Dynacast and its per-subscriber stream allocator/BWE own publication - layer activation and the actual layer forwarded to each SFU subscriber. - Screener does not implement a quality score, bandwidth estimator, periodic - layer controller, or per-Viewer encoder. -4. A subscriber's `HIGH` request is a ceiling, not a forced delivery layer. A - weak downlink may receive the lower representation while another subscriber - receives the highest available representation when the pinned LiveKit stack - can sustain that contract. -5. AdaptiveStream is a display-demand input, not a network detector. Current - Screener subscribers keep it disabled because any Viewer may become a relay - and LiveKit cannot see that Viewer's peer children. Network adaptation uses - the server stream allocator/BWE instead. -6. ADR-0005 remains the only route owner. Layer choice does not change topology, - endpoint capacity, SFU admission, or decoded-stall recovery authority. - -Production release `e14eb0e` left the screen-share layer list unset, so pinned -client `2.22.0` supplies a same-FPS half-resolution representation beside the -original. A live mixed P2P/SFU observation confirmed that the 60 fps lower -representation can consume enough Host-to-SFU budget to starve the highest -representation. The accepted correction keeps LiveKit's two-layer ownership but -supplies one half-resolution lower preset with -`lowFps = min(30, highFps)` and the SDK's existing proportional bitrate formula. -The two encodings remain RIDs `q,h`; LiveKit's protocol labels them `LOW,MEDIUM`, -while a subscriber's default `HIGH` ceiling still selects the highest available -encoding. Dynacast, server send-side BWE, and disabled AdaptiveStream remain -unchanged. Production release `be53c4d` deploys that correction. - -Current source derives the lower preset from the actual capture dimensions and -passes it through LiveKit's public publication options. Retained publication -options give LiveKit the same preset on its own source-replacement or republish -boundary; Screener does not mutate the active lower encoding behind LiveKit's -Dynacast sender lock. - -## Pinned Physical Result - -The 2026-08-25 gate used LiveKit server `1.13.5`, client `2.22.0`, Chrome 151, -a continuously changing VP8 source, and one full-resolution plus half-resolution -screen-share publication. Manual maximum/lower selection delivered `1280x720` -and `640x360` at about 30 fps, and the same connection recovered from the lower -to the full-resolution representation. - -With the server default receiver-side BWE, an approximately 0.96 Mbps subscriber -remained on HIGH for about 21 seconds and ended at zero decoded fps. Enabling -LiveKit `congestion_control.use_send_side_bwe` made the native stream allocator -select LOW; by the third steady window it delivered `640x360` at about 29 fps and -88 KB/s. A fresh unshaped subscriber immediately received HIGH. The constrained -connection's automatic upgrade after removing shaping was not isolated because -Chrome bound that test condition when creating the PeerConnection; manual and -AdaptiveStream same-connection LOW-to-HIGH recovery were independently proven. - -Dynacast paused layers with no subscribers after about five seconds. Lower-only -demand disabled the full-resolution encoding, while maximum demand kept both -encodings active; the second VP8 simulcast encode is therefore an accepted -bounded cost, not eliminated by Dynacast. AdaptiveStream selected the full -representation for a large attached element, the lower representation for a -small element, paused a hidden element, and recovered when visible. With no -attached element it received only the lower representation, confirming that it -cannot own relay ingress. This gate established the framework combination of -VP8 screen-share simulcast, Dynacast, server send-side BWE, and -`adaptiveStream: false`; the later production observation amends only the lower -representation's FPS and proportional bitrate. +1. Browser video uses VP8 across direct, browser-relay, and SFU paths. Codec + selection, backup codecs, runtime codec switching, and codec state in the + room wire are not product features. +2. Display video uses the standard `contentHint = "motion"`; display audio uses + `contentHint = "music"`. The hint expresses content intent and does not + promise a resolution, frame rate, bitrate, encoder, or hardware path. +3. The three recommended profiles and advanced controls supply capture and + sender ceilings plus `degradationPreference`. Every sender reads back the + parameters that the browser actually accepted. +4. Direct and peer paths leave media adaptation to each `RTCPeerConnection`. + After accepting an answer, the Host reapplies the current video profile to + the negotiated sender because the browser may replace or rewrite encoding + parameters during negotiation. +5. The SFU publisher sets VP8, the selected HIGH ceiling, and degradation + preference, but does not set `screenShareSimulcastLayers`, mutate lower + encodings, or select a subscriber layer. Pinned LiveKit defaults own the + representation set and source-replacement/republish behavior. +6. LiveKit Dynacast and server send-side BWE remain enabled. AdaptiveStream + remains disabled because a Viewer may relay its received track to peer + children; local DOM size or visibility cannot represent that downstream + demand. +7. Quality measurements are diagnostic. Bitrate, resolution, FPS, RTT, jitter, + loss, freeze counters, codec, and limitation reason do not trigger parent + selection, relay abdication, periodic rebalancing, or route changes. Only + hard connection failure, the existing non-paused decoded-frame stall, parent + departure, or capacity invalidation can make an active edge unusable. +8. A healthy decoded route remains sticky. Manual media reconnect rebuilds the + current exact P2P parent or current SFU subscription; it does not search for + a better parent. If current-route recovery genuinely exhausts, ADR-0005's + existing controller tries other eligible P2P parents before SFU. ## Evidence Boundary -The retained exact-production A/B used Chrome 151 with an always-active ordered -`q,h` publication. The 1904x928 `h` stream was about 9.2 fps / 1.37 Mbps with -both encodings active and about 23.3 fps / 3.32 Mbps with `q` inactive. Capture -remained near 60 fps, encode cost was about 4.46 ms per frame, loss and -retransmission were zero, and available outgoing bitrate was about 4.81 Mbps. -This rejects always-on `LOW` for that path. It does not establish Dynacast, -AdaptiveStream, heterogeneous subscriber behavior, or real-game performance. +Chrome 151 production measurements established three durable facts: -Quality evidence remains diagnostic. Track availability, bitrate, resolution, -FPS, RTT, jitter, loss, codec and current LiveKit layer cannot authorize route -mutation or predict another parent. +- direct and browser-relay VP8 can sustain about 60 fps after stock bandwidth + estimation warms up; sender ceilings are not startup guarantees; +- forcing an always-active lower SFU encoding can reduce the highest encoding + on the same Host-to-SFU congestion budget; and +- pinned LiveKit `2.22.0` plus server `1.13.5` can select lower and higher VP8 + screen-share representations per subscriber when server send-side BWE is + enabled. + +Separate content-hint probes showed that `motion` changes Chromium's adaptation +tradeoff and may spatially downscale to preserve motion. That is the standard +behavior requested for games, not a quality floor. Real-game readability, +weaker Hosts, heterogeneous devices, and public-network SFU quality still +require physical evidence. ## Consequences -- A weak SFU Viewer can receive a real lower spatial representation when the - framework selects it; the SFU does not transcode. -- The Host publishes at most two SFU representations regardless of Viewer - count. Dynacast pauses representations above aggregate demand and all - representations when no subscriber remains; maximum demand keeps both VP8 - simulcast encodings active. -- Host encode capacity is handled through the existing explicit share profiles, - not by falling back to one encoding and abandoning constrained subscribers. -- An SFU-fed relay keeps a subscription ceiling sufficient for its subtree; - local DOM visibility alone cannot pause that ingress. -- A configured 60 fps or bitrate remains a ceiling, not a delivery guarantee. +- Screener owns fewer media mechanisms and follows the pinned frameworks' + supported control surfaces. +- A constrained Viewer may receive a lower LiveKit representation without + lowering every subscriber, subject to the publisher and network actually + sustaining the framework contract. +- A Host may need to lower its explicit share profile when encoder or uplink + capacity is insufficient; Screener does not silently remove constrained + Viewer support. +- Quality-based topology optimization remains unimplemented. Accepting it later + requires evidence for alternative-path measurement and a separate route-model + decision. ## Stop Lines -- Do not create one encoder or representation per Viewer. -- Do not add application quality scoring, periodic polling, manual layer - switching, or quality-driven reparenting. -- Do not use AdaptiveStream without the LiveKit track attachment contract. -- Do not infer Dynacast success from configuration or source inspection alone. -- Do not apply SFU layer policy to ordinary direct or peer paths. +- No Screener resolution/FPS/bitrate ladder or representation formula. +- No manual SFU layer selector, forced single HIGH publication, or per-Viewer + encoder. +- No quality score, all-pairs probing, periodic rebalancing, or speculative + parent switching. +- No AdaptiveStream while a subscriber can relay the track. +- No quality or hardware claim from configured options alone. ## References -Primary sources checked through 2026-08-25: - +- [MediaStreamTrack Content Hints](https://www.w3.org/TR/mst-content-hint/) +- [WebRTC](https://www.w3.org/TR/webrtc/) +- [WebRTC Statistics](https://www.w3.org/TR/webrtc-stats/) - [LiveKit video simulcast and Dynacast](https://docs.livekit.io/transport/media/advanced/) - [LiveKit selective subscription](https://docs.livekit.io/transport/media/subscribe/) -- [LiveKit client 2.22.0 room options](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/options.ts) -- [LiveKit client 2.22.0 AdaptiveStream track attachment](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/track/RemoteVideoTrack.ts) -- [LiveKit client 2.22.0 subscriber quality ceiling](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/track/RemoteTrackPublication.ts) -- [LiveKit client 2.22.0 Dynacast handling](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/participant/LocalParticipant.ts) -- [LiveKit server 1.13.5 per-subscriber layer application](https://github.com/livekit/livekit/blob/v1.13.5/pkg/rtc/subscribedtrack.go) -- [LiveKit server 1.13.5 Dynacast quality aggregation](https://github.com/livekit/livekit/blob/v1.13.5/pkg/rtc/dynacast/dynacastqualityvideo.go) -- [LiveKit server 1.13.5 enabled-quality generation](https://github.com/livekit/livekit/blob/v1.13.5/pkg/rtc/dynacast/dynacastmanagervideo.go) +- [LiveKit client 2.22.0 encoding construction](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/participant/publishUtils.ts) +- [LiveKit server 1.13.5 layer forwarding](https://github.com/livekit/livekit/blob/v1.13.5/pkg/sfu/forwarder.go) diff --git a/docs/deployment.md b/docs/deployment.md index 9efe5aa..d7090a3 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -3,7 +3,7 @@ Last verified against upstream documentation: 2026-08-24. This page records production running exact deployed application/runtime revision -`be53c4d5794d38d5d406c876309958d7117ee601`, release `be53c4d`, and the single +`af348ee1d508a3af02b18a7f46c461953798e19d`, release `af348ee`, and the single Browser `screener-v12` contract. Product direction and pending work are owned by [project memory](./project-memory.md) and [the TODO ledger](./todo.md). @@ -13,9 +13,8 @@ Caddy or nginx; ordinary peer ICE advertises only STUN, while Browser LiveKit PCs configure no external ICE server and retain LiveKit-signaled UDP candidates. LiveKit supplies bounded SFU capacity. Normal media remains distributed through direct or peer edges whenever those paths work. Production includes -the Browser SFU ICE isolation and two-representation model. Chrome 151 physically -verified UDP publication/subscription and native full/lower selection; the -deployed lower-representation cost correction still needs mixed-route media evidence. +Browser SFU ICE isolation and leaves representation construction and selection +to pinned LiveKit with Dynacast and server send-side BWE. A deployment may additionally provide one dedicated single-node LiveKit process as the current controller's automatic final media fallback. This capacity is @@ -26,21 +25,21 @@ ICE/UDP only. Ordinary peer ICE remains STUN-only and production coturn uses the tracked STUN-only configuration with TCP/TLS disabled. The source and production configure no TURN, ICE/TCP, media TCP, or TLS-relayed media. -Production runs exact `be53c4d5794d38d5d406c876309958d7117ee601`, release -`be53c4d`, from `/opt/screener/releases/be53c4d`. The immutable runtime tar +Production runs exact `af348ee1d508a3af02b18a7f46c461953798e19d`, release +`af348ee`, from `/opt/screener/releases/af348ee`. The immutable runtime tar SHA-256 is -`5c174c30ffacd98738a364b825b915b84432823c261272684937361be2451144`. +`4922cc31d7caaad7c5412b1f2fe1d73f788d16d92386efad6eac358a97b8a78e`. Its 39-file path/size/hash manifest SHA-256 is -`c34dba43a25c6b8447e1153a5be3191bf82cff94117ed31edc325d3f7cf8541f`. +`5b397809b664b9a55885ac603df615b537bf51276e7274416820870956242d7d`. Local and public `/healthz` return 200; Screener, LiveKit, coturn, and nginx are active with `NRestarts=0`. The public main Browser asset is -`assets/index-BKEVI7Ux.js` with SHA-256 -`22114fc8a26b3f03f37973555433d1e73d52d1c7b76fa3c7cc597500823bd62d`. +`assets/index-BvuZY6Hc.js` with SHA-256 +`e8fc0500496ee101031a9cfebfcc22d3b69e66d2cac5e875448b56620dbb21c5`. The release deploys the single Browser `screener-v12` wire, fixed VP8 for Browser -direct, browser-relay, and SFU video, one original plus one half-actual-resolution -SFU encoding capped at 30 fps with proportional bitrate, Dynacast and server -send-side BWE without an application layer selector, no video `contentHint`, no +direct, browser-relay, and SFU video, pinned LiveKit default screen-share +representations, Dynacast and server send-side BWE without an application layer +selector, video `contentHint = "motion"`, no codec UI/state/wire, random four-digit memory rooms with a 24-hour dormant lease, a room-lived 22-character grant, orthogonal `open | private` grant/code admission, @@ -284,7 +283,7 @@ lifetime. An actively connected Host prevents expiry; explicit stop or Host disconnect starts the lease, and only the exact Host token renews it before expiry. Viewer activity never renews ownership. `ROOM_DATABASE_PATH` and `ROOM_TTL_SECONDS` fail startup even when blank. -Production `be53c4d` accepts 1 through 20 and explicitly selects 20. This is an +Production `af348ee` accepts 1 through 20 and explicitly selects 20. This is an admission limit, not evidence that every publisher, network, or quality profile can sustain that many streams. `ENDPOINT_MEDIA_COPY_CAPACITY` defaults to 2 and accepts only 1, 2, or 3. It is @@ -297,7 +296,7 @@ must fail or wait before a fourth endpoint copy is issued. Supplying the removed `MAX_PEER_RELAY_DOWNSTREAM_EDGES`, even blank, fails startup. -Production release `be53c4d` runs the deployed server and Browser assets +Production release `af348ee` runs the deployed server and Browser assets atomically on `screener-v12`; every stale Browser or executable-sender wire fails before room authority. Native senders and helpers are outside this release. diff --git a/docs/project-memory.md b/docs/project-memory.md index 0f8488a..f24525c 100644 --- a/docs/project-memory.md +++ b/docs/project-memory.md @@ -4,56 +4,127 @@ Last updated: 2026-08-25 ## Current Product Truth -- Build private, low-latency game screen sharing for one broadcaster and a small group of trusted friends. Public or large broadcasts belong on OBS/Twitch-class services. -- A room supports one Host and up to `20` authenticated Viewers. Admission is independent of endpoint fanout and SFU capacity; the event-driven route controller must remain efficient at that bound without all-pairs probing. -- Authenticated Hosts and shipped clients are trusted media participants in these private rooms; route authority, generation fencing, and server admission remain fail-closed. -- Rooms use one bounded, process-memory model: a random free code from `1000..9999`, default 24-hour configurable dormant lease, and no database or cross-restart recovery. Active sharing never expires; stop/disconnect starts the lease, the exact Host token may resume before expiry, Viewer activity does not renew it, and expiry releases the code. Process restart invalidates every room, token, invitation, password verifier, and participant. -- Viewer grants and code entry are orthogonal. Each room has one random 128-bit/22-character grant digest bound to that exact in-memory incarnation; it has no independent expiry and ends on reclamation, restart, rotate, or revoke. Every new room creates that invitation and defaults code entry to public `open` with no password; the only other code-entry state is `private`. A private room with no password is invitation-only, while setting a password also permits matching code-only entry. An unallocated/reclaimed/expired code returns `ROOM_NOT_FOUND` without a password prompt, while other existing-room denials remain `ROOM_ACCESS_DENIED`. The Host browser retains its creation preferences and raw ownership token; exact-token invitation/policy/password management remains available while dormant without starting media or renewing the lease. The server keeps only current-process digests/verifiers and uses no fingerprint. Recreation reapplies preferences but creates a new code and credentials. -- Web is the current delivery target. The route release covers Browser Host, Viewer, and relay endpoints; packaged/native senders, capture helpers, shared-encode executables, and their acceptance runs are lowest-priority later work outside this milestone. -- Keep routing automatic and media distributed. Direct/peer UDP is preferred; the dedicated LiveKit SFU/UDP path is the only application fallback and is not the default topology. Screener configures no TURN, ICE/TCP, media TCP, or TLS-relayed media path. -- Every non-server endpoint uses one server-authoritative steady outbound media-copy capacity: default `2`, statically configurable as `1`, `2`, or `3`. A peer child or the Host publication consumes one slot; upstream receive is free, and SFU subscriber egress is accounted at the server. Browser role/UA/visibility does not create a separate tier, and clients cannot raise the deployment value. -- The stable routing invariants are one authoritative upstream per Viewer, an acyclic source-reachable active graph, exact authenticated route attempts, endpoint and server admission, one room-serial child operation, and bounded success, wait, or failure. -- One room controller reconciles allocation, child reparenting, and relay drain through the same operation. It finds one waiting Viewer or invalid edge and builds one deterministic tuple list. The operation owns its cursor, current candidate and reservations, fact version, plus one total deadline; only its current candidate may produce media. A prepare names that exact child, selected transport, and server-issued candidate connection ID. A usable old edge remains until the exact candidate child decodes its first new video frame. `settleReady` synchronously commits admission before graph promotion; success commits prepare revision `P`, while failure releases resources, broadcasts rollback `R > P` from one monotonic allocator, and advances the cursor without resetting the deadline. Exhaustion retires hard-invalid, departed-parent, or overflow edges into explicit waiting but retains a healthy bootstrap edge. Bounded-gap preflights server resources, replans the untried suffix after retirement, and appends one ordinary restore candidate when it cut healthy media. A relay with a bad ingress reparents itself as a child and retains its subtree. A disconnected endpoint, or one whose effective downstream capacity falls below its current child count, accepts no excess children; its direct or overflow children enter the same operation and a disconnected endpoint is removed when childless. Authoritative pause aborts the operation, suppresses decoded-progress stall decisions, and resumes from a fresh reconciliation. -- One Host publication may serve all SFU subscriptions. Its exact generation owns one ingress handle and exact Viewer subscription handles own egress; later SFU children reuse the publication, while reserved, committed, and draining handles remain charged until the managed room is deleted and proven absent. Actual SFU usage release wakes only rooms that previously lost a candidate to global admission, and SFU-fed and peer-fed endpoints use the same provisional-child transaction. Parent choice hard-filters authority, reachability, cycles, effective capacity, and producer transition, then orders by shallowest result, remaining steady sender capacity, stable join order, and peer identity; server admission is checked at the current cursor. Acyclicity and room admission bound maximum depth; depth is observed but has no separate hard cap. Healthy committed edges remain sticky: the controller repairs waiting, invalid, departed-parent, and overflow children but does not periodically rebalance the graph. Raw IP, claimed NAT type, UA, geography, all-pairs probes, weighted scores, and periodic room rebalancing are not route inputs. -- Ordinary peer ICE is STUN-only. Browser LiveKit publisher and subscriber PCs use no external ICE server, retain LiveKit-signaled UDP candidates, and let standard ICE nominate a non-relay pair; deployment STUN remains available to ordinary peers and LiveKit server-side public-IP discovery. This Browser SFU ICE-server isolation avoids the reproduced external-STUN/SFU association conflict without constraining candidate type or count and without adding TURN, another route, a port, or a timeout. Credentials, room secrets, candidate details, and diagnostic data remain private and narrowly scoped. A future strict-firewall transport requires separate evidence and must remain inside LiveKit rather than becoming another Screener route candidate. -- Browser peers use existing periodic WebRTC/LiveKit sampling to maintain one exact-route decoded-progress timestamp. New frames or route/connection changes reset it; a non-paused deadline reports once. An active SFU cadence treats an absent track/report or stats failure as no progress without fabricating metrics and stops only on deactivation/teardown. Other current-path stats remain diagnostic and cannot prove another parent better. -- ADR-0007's two VP8 SFU representations, Dynacast, server send-side BWE, and disabled AdaptiveStream for potential relays are deployed. The lower representation is half the actual capture dimensions, capped at 30 fps, and uses proportionally derived bitrate. Screener adds no media-layer selector. -- Browser capture requests available share audio by default on initial capture and source switch, with an explicit video-only warning when no audio track is returned. Its screen-audio control offers live-switchable 64/128/256 kbps sender ceilings and defaults to 128. The last-wins quality wire owns desired state; each endpoint serially applies and reads back current Host/relay/SFU audio senders, exposes local failures, and gives future senders the latest desired value. Opus remains fixed, and without an applied-ack wire the Host does not claim room-wide atomic convergence. These are configuration ceilings, not fidelity claims. -- Browser display video leaves `contentHint` unset so Chromium retains its screen-capture classification; returned audio tracks keep `contentHint = "music"`. Browser video is fixed to VP8 across direct, browser-relay, and SFU paths; codec selection, fallback to another media codec, quality-state/wire fields, and share-lifetime codec branches are not product features. Actual codec/profile/encoder stats remain diagnostic only. -- Recommended quality remains exactly `1080p60`, `1080p30`, and `720p30`, with the middle `1080p30` as the accepted default. `480p` means only an advanced `854x480` resolution choice whose FPS and bitrate remain independently selected; it is not a fourth recommended profile. -- Two distribution artifacts are accepted only after the current feature, NAT, and real-network ledger is complete. A public-server one-click deployment package installs the current application plus STUN/SFU and proxy components on a user-owned server. A Windows/macOS/Linux local package runs the Host, application server, and local state on the broadcaster's machine without source, Node, or any external Screener/network service; it keeps workable direct P2P but reports the reachability loss imposed by public ingress, TLS, gateway, NAT, or firewall limits rather than promising universal connectivity. -- Current UI, presence, route labels, and diagnostics describe observed state only; they do not create route authority. -- Before an explicit public-release decision, the private deployment and repository keep only the current product contract. Incompatible wire, configuration, parser, and API changes ship atomically; stale clients fail before authority, replaced compatibility surfaces are deleted, and Git history owns rollback. -- A routine application-only deployment verifies one new immutable artifact and switches to it atomically while reusing unchanged environment, service, LiveKit, coturn, reverse-proxy, and firewall configuration. The pre-cutover application release is guaranteed only through bounded health and postflight checks; it has no retention contract afterward and is not a maintained backup. A future change to infrastructure, secrets, persistent data, or irreversible state must define recovery only for the surfaces it actually changes. -- Viewer presentation stays on stage until current-generation frame proof, distinguishes typed access/Host/route/media/autoplay/recovery phases, and reserves `Play` for autoplay rejection. Pages expose local connection details but no diagnostic download; the bounded Host-only route snapshot remains for acceptance tooling. Route/quality facts are diagnostic only. Browser ICE owns direct reachability; Screener does not predict ports, classify NATs, probe TCP, synthesize candidates, or reparent healthy edges speculatively. +- Screener is private, low-latency game screen sharing for one Host and up to + `20` authenticated Viewers, not a public broadcast service. +- Web Host, Viewer, and Browser relay are the current delivery surface. Native + capture, shared encoding, and distributable server/local packages remain + later work. +- Rooms are bounded process memory: random free `1000..9999` code, default + 24-hour dormant lease, active sharing never expires, exact Host token resume, + Viewer activity does not renew, and process restart loses every room and + credential. There is no database or migration path. +- Every room has one 128-bit/22-character Viewer grant bound to that exact room + incarnation and independent `open | private` code entry. Private without a + password is invitation-only; adding a password also permits matching code + entry. Reclamation, restart, rotate, or revoke ends the grant. Missing codes + return `ROOM_NOT_FOUND` without a password prompt. +- Host creation preferences and raw ownership token stay local to the Host + browser. Server authority stores only current-process digests/verifiers and + never uses IP, UA, device, or browser fingerprint as identity. + +## Media And Routing + +- Media stays automatic and distributed. Ordinary peer ICE is STUN-only and + direct/peer UDP is preferred. The only application fallback is the dedicated + LiveKit SFU over UDP; Screener configures no TURN, ICE/TCP, media TCP, or + TLS-relayed media. +- Browser LiveKit publisher/subscriber PCs use no external ICE server and retain + LiveKit-signaled UDP candidates. Deployment STUN remains available to ordinary + peers and LiveKit server-side public-IP discovery. +- Every non-server endpoint uses one server-authoritative steady outbound copy + cap: default `2`, configurable only as `1`, `2`, or `3`. A peer child or the + Host publication consumes one slot; upstream receive is free; SFU subscriber + egress uses separate server admission. +- One room controller owns one committed acyclic source-reachable graph, one + event-driven reconciliation loop, and at most one room-serial child operation. + The operation owns one deterministic candidate list/cursor, one current + candidate and reservations, one fact version, and one total direct-then-SFU + deadline. +- Candidate filtering enforces current authority, reachability, acyclicity, + capacity, transition slots, and server admission. Eligible P2P parents are + ordered by resulting depth, remaining capacity, stable join order, and peer + identity before SFU. +- A candidate commits only after the exact child decodes its first new video + frame. Failure releases resources, broadcasts a strictly newer rollback + revision, advances the cursor without resetting the deadline, and preserves + unaffected healthy branches. A relay with bad ingress reparents itself while + retaining its subtree. +- Healthy decoded routes remain sticky. There is no periodic rebalancing, + quality score, all-pairs probing, parent blacklist, NAT classification, port + prediction, or independent depth cap. Quality evidence cannot prove an + unconnected parent would be better. +- P2P and SFU first use their framework reconnect behavior. P2P recovery tries + the same parent, and SFU recovery rebuilds the same publication/subscription. + Manual media reconnect also stays on the current exact route. Only actual + recovery exhaustion, hard failure, non-paused decoded-frame stall, parent + departure, or capacity invalidation enters route reassignment. +- Active-path WebRTC/LiveKit sampling owns one exact-route decoded-progress + deadline. Route/connection change or new progress resets it; authoritative + pause suppresses it. Page resume rebaselines frozen wall-clock time before + another no-progress sample can fail a route. + +## Media Quality + +- Browser video is VP8 only. Display video uses `contentHint = "motion"` and + display audio uses `contentHint = "music"`. Codec UI/state/wire, backup media + codecs, and runtime codec switching do not exist. +- Recommended profiles remain exactly `1080p60`, `1080p30`, and `720p30`, with + `1080p30` default. `480p` is only an advanced `854x480` resolution choice; + advanced FPS and bitrate remain independent. +- WebRTC owns direct/peer media adaptation. The Host reapplies the selected video + profile after answer negotiation; no periodic application controller exists. +- The SFU publisher sets VP8 and the selected ceiling but no custom simulcast + layers. Pinned LiveKit defaults own representations, Dynacast owns aggregate + demand, and server send-side BWE owns subscriber forwarding. AdaptiveStream + stays disabled because any Viewer may relay its received track. +- Screen audio requests capture by default and offers live 64/128/256 kbps + sender ceilings with 128 default. Peer answers request Opus stereo with a + 256 kbps receive maximum. SFU publication uses stereo, DTX off, and RED off; + disabling RED accepts reduced burst-loss resilience in exchange for bounded + publisher traffic. +- Configured resolution, FPS, bitrate, preference, codec, and audio ceiling are + requests or ceilings. Sender readback and RTCStats are the observable truth. + +## Presentation And Lifecycle + +- Viewer presentation is revision/generation-fenced and keeps the stage stable + until a current-generation composited frame. A new current frame clears stale + media `connecting/reconnecting` presentation; old callbacks cannot prove a + new route. +- Viewer local play/pause, volume, mute, and fullscreen are owned by native video + controls on one persistent media element. Host preview is a muted, control-free + view of the capture stream; only explicit Host actions pause or stop sharing. +- Hiding or unfocusing the Host page pauses only the local preview element. + Browser/OS capture and background behavior remain platform capabilities; no + fake keepalive, silent media, Wake Lock, or timer loop is a product mechanism. ## Current Source And Production -- Production runs exact application/runtime revision `be53c4d5794d38d5d406c876309958d7117ee601`, release `be53c4d`, on strict `screener-v12`; canonical `main` contains that deployed application tree plus current truth. -- Current source and production admit up to `20` Viewers and implement the uniform `1/2/3` endpoint cap, exact prepare candidate, first-decoded-frame commit, strict rollback revision, decoded-progress stall, one direct/SFU operation deadline, and one event-driven controller. Active-SFU sampling remains live through an absent track/report or stats failure. Ordinary peer ICE is STUN-only and the sole application fallback is SFU/UDP. Exact SFU admission and empty Browser SFU ICE-server options remain unchanged. -- Source and production use random four-digit memory rooms, dormant leases, exact-Host resume, restart loss, local preference replay, orthogonal grants/code entry, and no SQLite runtime. V12 uses a 22-character room-incarnation grant, `ROOM_NOT_FOUND`, no diagnostic-file export, and a compact UI showing the complete shorter URL. -- Exact production artifacts, services, ports, and any change-scoped infrastructure recovery boundary are owned by [deployment](./deployment.md) and indexed by [status](./status.md). -- Source and production require v12, fixed VP8, and no video hint, and derive a half-actual-resolution SFU lower preset capped at 30 fps with proportional bitrate. Both retain Dynacast, server send-side BWE, and disabled AdaptiveStream. Other current media and UI behavior matches the truths above. -- Browser SFU subscribers reconcile eligible Host screen publications across connect/activate event ordering, participant/track arrival, and reconnect. Production LiveKit logs only warn/error, and the application service has a bounded same-host LiveKit readiness preflight. -- The deployed application tree passed TypeScript, 613 Web tests, both builds, hygiene, two independent reviews, immutable artifact verification, and production postflight. Earlier exact-production gates closed direct/browser-relay VP8 cadence, active-audio ceilings, a single-representation SFU baseline, and one controlled exact-candidate SFU commit; the pinned-stack gate proved native representation selection. The lighter representation's mixed-route quality effect, controlled recovery on this release, representative networks, all-UDP-blocked failure, public SFU 60 fps, and real-game synchronization remain unproven. Exact metrics live in [verification status](./verification-status.md). -- Chrome loopback, cap2/cap3, short resource, recovery, MBB, and A/V fixtures are bounded evidence only. They do not establish quality, resource limits, or product policy. +- Canonical `main` and production run exact application/runtime revision + `af348ee1d508a3af02b18a7f46c461953798e19d`, release `af348ee`, on strict + `screener-v12`. +- Production uses random memory rooms, 20-Viewer admission, endpoint cap `2`, + dedicated LiveKit admission `1` ingress / `20` egress, STUN UDP 3478, LiveKit + media UDP 7882, and Web TCP 80/443. Node 8787 and LiveKit 7880 remain private. +- Exact artifact and service state are owned by [deployment](./deployment.md) + and indexed by [status](./status.md). Physical evidence boundaries are owned + by [verification status](./verification-status.md). ## Current Priority -1. Verify the deployed lower-representation quality effect on a mixed P2P/SFU room, then repeat controlled active-SFU recovery and finish real-network route, screen-audio, and real-game A/V acceptance. VP8 acceleration, mobile lifecycle, Native, distribution, and repository simplification remain later. +1. Integrate and deploy the accepted Viewer lifecycle/native-control/current- + route reconnect candidate, then finish representative real-network route, + screen-audio, real-game A/V, and mobile lifecycle acceptance. -The desktop Host background/minimized report is deferred until it reproduces on current production with a real game and synchronized media plus CPU/GPU evidence. The bounded current-Chrome screening found no immediate Host-page lifecycle drop, and removal of the old harmful `contentHint = "motion"` remains a plausible confound rather than a proven background fix. Mobile Viewer playback and assigned-relay survival remain a separate lifecycle gate. +Quality-driven parent selection, VP8 hardware evidence, Native sender work, +distribution packages, whole-product UI/bilingual polish, and repository-wide +simplification remain later decisions in [the TODO ledger](./todo.md). ## Working Rules -- Generic task authority, scope, smallest-correct implementation, and current-result delivery are owned by the repo-tracked [`stop-that-shit` skill](../.agents/skills/stop-that-shit/SKILL.md); repository-specific truth and execution rules remain in `AGENTS.md` and `CONTRIBUTING.md`. -- Old branches may contribute scoped code or evidence only after reconciliation. Their memory, requirements, ADRs, status, and deployment snapshots never overwrite newer mainline truth. -- Preserve dirty, unique, open-stack, and evidence worktrees until their disposition is explicitly decided. Workspace counts are sampled on demand rather than stored here. - -## Source Map - -- Current execution: [status](./status.md) -- Executable and held work: [TODO ledger](./todo.md) -- Requirements and design: [requirements](./%E9%9C%80%E6%B1%82%E7%90%86%E8%A7%A3.md), [design](./%E6%96%B9%E6%A1%88%E8%AE%BE%E8%AE%A1.md), and [ADRs](./adr/) -- Evidence and limits: [research](./research/) and [verification status](./verification-status.md) -- Operations: [deployment](./deployment.md), [maintenance](./maintenance.md), and [contributing](../CONTRIBUTING.md) +- Generic authority and scope follow the repo-tracked + [`stop-that-shit` skill](../.agents/skills/stop-that-shit/SKILL.md). +- Canonical `main` and the current truth owners override old branches, + worktrees, handoff text, and chat summaries. +- Preserve unique or dirty candidates until integration and reference/reparse + audits prove cleanup is safe. diff --git a/docs/research/browser-background-capture.md b/docs/research/browser-background-capture.md index bd00f96..b1e51a4 100644 --- a/docs/research/browser-background-capture.md +++ b/docs/research/browser-background-capture.md @@ -3,9 +3,8 @@ - Research date: 2026-08-24 - Scope: desktop Chrome/Edge Web Host capture while the Host page is unfocused, occluded, backgrounded, or minimized -- Status: deferred until a current-production real-game reproduction exists; - one bounded current-browser screening did not observe an immediate Host-page - background drop, and no new fix is accepted, implemented, or deployed +- Status: Host capture report remains unconfirmed; Viewer resume/state lifecycle + correction is accepted pending integration and does not claim capture keepalive ## Current Conclusion @@ -53,34 +52,29 @@ configuration. Encoder implementation is a diagnostic variable only after a stable baseline exists; neither codec nor encoder choice is a page-keepalive mechanism. -The 2026-08-19 report came from exact release `769de201f7cc`, which still set -video `contentHint = "motion"`. Current source and production leave the video -hint unset; a separate controlled VP8 probe changed from 14.93 fps at 428x208 -with `motion` to 29.73 fps at 1904x928 without it. That known old-source defect -is fixed in the current contract, but it does not prove that the state-dependent -background report had the same cause. The physical baseline must therefore -start from current production rather than carrying the old report forward as a -current regression. +The accepted Browser path uses VP8 and `contentHint = "motion"`. A controlled +probe showed that this standard game-motion hint can spatially downscale to +preserve cadence; that quality tradeoff is independent from page lifecycle and +does not explain a state-dependent background report. The physical baseline +must start from current production rather than carrying an old report forward +as a current regression. -This investigation is deferred. It reopens only if the report -reproduces on current production with a continuously changing real game and -synchronized capture, send, receive, decode, CPU, and GPU evidence. Removing the -old harmful `motion` hint may have removed a confounding quality defect, but it -is not established as a background-lifecycle fix. Mobile Viewer background -playback and assigned-relay survival remain a separate browser/OS lifecycle -gate and do not inherit this desktop Host result. +The Host investigation reopens only if the report reproduces on current +production with a continuously changing real game and synchronized capture, +send, receive, decode, CPU, and GPU evidence. Mobile Viewer background playback +and assigned-relay survival remain a separate browser/OS lifecycle gate and do +not inherit this desktop Host result. ## Current Source Boundary -The no-video-hint review baseline is -`a26eb39dc677003110787b0ed1581c208f894fd7`. Current source and production use -fixed VP8, no video hint, and no codec UI, quality state, or wire field; their -exact identity is owned by [status](../status.md). +Current source and production use fixed VP8, video `contentHint = "motion"`, +audio `contentHint = "music"`, and no codec UI, quality state, or wire field; +their exact identity is owned by [status](../status.md). The current path does not claim or implement a page-keepalive mechanism, and deployment health supplies no physical background-capture evidence. - Current `src/client/media/quality.ts` obtains one `getDisplayMedia()` stream, - applies ideal/max capture constraints, leaves the video hint unset, and + applies ideal/max capture constraints, sets standard game-motion intent, and applies sender bitrate, frame-rate, and degradation ceilings. Explicit sharing pause and authoritative reconnect re-pause change capture tracks' `enabled` state; neither manufactures foreground activity. Audio @@ -102,6 +96,10 @@ deployment health supplies no physical background-capture evidence. - The short-lived pending-candidate decoded-frame observer reads cumulative RTP progress only for the exact pending route. It neither drives capture nor proves document activity, and it stops when that operation settles. +- Viewer resume/visibility recovery rebaselines the existing decoded-stall wall + clock and rearms current-frame presentation. It prevents frozen JavaScript + time from becoming an immediate route failure; it does not keep a page, + capture source, encoder, or network process alive. - `scripts/peer-assisted-benchmark.ts` is not evidence for this issue: its synthetic canvas source is timer-driven and its browser launch explicitly disables background timer, occluded-window, and renderer backgrounding. @@ -187,7 +185,8 @@ All sources were checked on 2026-08-24. ## Bounded Current-Browser Screening -On 2026-08-24, Chrome `151.0.7922.174` on Windows 25H2 build `26200.9168` +On 2026-08-24, before the current motion-hint decision, Chrome +`151.0.7922.174` on Windows 25H2 build `26200.9168` captured a continuously changing native WPF window through real `getDisplayMedia()`. The source remained visible and changing; capture returned 1186x712 at a 30 fps setting. One VP8/libvpx sender used no video hint, a 5 Mbps @@ -250,8 +249,8 @@ them directly. Open `chrome://webrtc-internals` on Host and Viewer before sharing and keep its overhead constant across every run. Leave audio-debug recording, event-log recording, packet capture, and media recording disabled. The current Screener -diagnostic download may be taken at state boundaries as a latest-sample -cross-check; it is not used as a time series. If a stable reproduction suggests +connection-details view may be used as a latest-sample cross-check; it is not a +time series and has no diagnostic download. If a stable reproduction suggests freezing or discard, inspect the Host tab from a pre-opened separate control window in `chrome://discards` or `edge://discards` outside the timed measurement window. That page may expose URLs and therefore remains local evidence. diff --git a/docs/research/browser-screen-audio-quality.md b/docs/research/browser-screen-audio-quality.md index 9a19d61..753c969 100644 --- a/docs/research/browser-screen-audio-quality.md +++ b/docs/research/browser-screen-audio-quality.md @@ -1,551 +1,127 @@ -# Browser Screen-Audio Quality Controls - -Accessed: 2026-08-25 - -Status: current Browser source and production use strict `screener-v12`, fixed -VP8, no video hint, and one Opus screen-audio path with bounded 64/128/256 kbps -sender ceilings. Exact source and production identities are owned by -[status](../status.md). The capture request disables local speech processing, -live mutation has applied readback, relay track changes preserve the persistent -stream, and SFU reconnect reacquires exact publications/senders. -Exact-production `bf32859` physical evidence closes the active-source `1 kbps` report; -audible game quality and real-game SFU A/V synchronization remain open. - -## Observed-Bitrate Identity Boundary - -The interval calculation `(bytes * 8) / milliseconds` already returns kbps, and -the UI rounds that observed value rather than displaying a configured ceiling. -Current source binds ordinary and SFU publisher stats to the exact active audio -sender, merges the SFU sender reports, resets identity-changing baselines, keeps -late relay audio changes on exact children, and reacquires exact SFU senders on -reconnect. Explicit no-audio remains unknown. - -Pinned LiveKit 2.22.0 already republishes screen tracks after a full reconnect -from the saved publication options, including the audio preset; a signal-only -resume does not replace the sender. Screener must still reacquire its own exact -publication/sender references, reset identity-bound stats, fail closed when an -expected republish is missing, and propagate same-stream audio changes to its -downstream peer children. Reapplying unchanged audio sender parameters after -every `Reconnected` event duplicates the SDK on stable reconnects. A later -cleanup may skip that write when the sender identity and desired readback already -match, while retaining saved-option updates and the latest-profile race check. - -A Chrome 151 gate against exact production `bf32859` used real -`getDisplayMedia()` with generated active audio. Two Host-direct children, one -browser-relay child and one SFU Viewer all advanced audio energy and video frames -at each ceiling and after source replacement; the active UI never remained at -`1 kbps`. Direct/browser-relay traffic tracked the selected ceiling. SFU wire -traffic exceeded the 64/128 ceilings because the accepted pinned LiveKit -publication retains RED; this does not change the exact sender parameter -readback. Ordinary-route audio-minus-video playout timestamps ranged from -101 -to +4 ms once RTCP mapping arrived. Chrome exposed no such field on the SFU -subscription, which remains unknown rather than zero or inferred. A muted or -idle Opus stream may still legitimately use about 1 kbps. - -## Scope And Decision - -This review covers audio returned with `getDisplayMedia()` and sent through a -browser `RTCPeerConnection`. It does not decide whether the product ultimately -needs Windows per-application capture; that remains a separate native-sender -requirement. - -The product keeps one screen-media audio mode and exposes only three bounded -sender ceilings rather than codec plumbing: - -- request audio with echo cancellation, noise suppression, automatic gain, and - voice isolation disabled, while preferring two capture channels; -- request window audio for a selected window and offer system audio for a full - display; -- retain `contentHint = "music"` only as source intent, with no quality claim; -- treat an audio track as optional and warn before publishing when none exists; -- preserve returned audio through source changes, picture pause, P2P, relay, and - configured SFU routes; -- offer 64/128/256 kbps sender ceilings, default 128 kbps, with one 256 kbps - Opus receive maximum and DTX off across Web peer and SFU routes; -- let the browser's existing WebRTC congestion control reduce actual audio and - video traffic; do not add an application audio adaptation loop; -- expose no channel-count, sample-rate, codec, stereo, DTX, RED, FEC, arbitrary - bitrate slider, or application-owned adaptation control; and -- expose local, read-only audio RTP diagnostics without treating negotiated - codec fields as source-quality facts. - -The music hint is metadata, not a codec or quality mode. Chromium accepts and -reads it back, but the inspected libwebrtc audio-track interface has no matching -content-hint input to the Opus encoder. Screener therefore does not credit the -hint with a quality change. RFC 7587 and the pinned LiveKit presets, rather than -an ordinary-PC benchmark, define the 128 kbps default: the RFC places full-band -stereo music in a 64--128 kbps sweet spot, and LiveKit 2.22.0 names 128 kbps -`musicHighQualityStereo`. Requested, applied, negotiated, and observed states -remain separate. - -The advanced panel is named Share advanced settings. Current source stores the -64/128/256 choice and applies that sender `maxBitrate` ceiling on the existing -Opus path when it creates a sender. A change during an active share serializes -`getParameters()`/`setParameters()` updates and readback, keeps media and the old -applied ceiling on failure, and does not claim room-wide -convergence without a remote applied acknowledgement. This does not require -audio codec renegotiation. On the declared Chrome/Edge screen-audio Host -baseline, pinned LiveKit 2.22.0 can update an existing audio sender without -republish. Firefox's initial publish path may also write the preset into Opus -fmtp, so a sender-only increase is not claimed there until real readback and -receive evidence pass. A future microphone/voice feature remains a separate -track and processing path with its own -AEC/noise-suppression/DTX contract. It must not turn movie or game audio into a -voice-processed source. - -## Deployed Reports And Echo Boundary - -Users first reported that production -`6ccb516a47261054f91dfa2fafa408d39ced59fc` sounded poor for movie/video screen -audio. Current source and production use peer/SFU stereo and the matching -`maxaveragebitrate=128000`, but the speech-gated sound remained. A -phone Viewer connected directly through the SFU reproduced it, so browser-relay -decode/re-encode is a route-specific amplifier rather than the common cause. -The shared Host capture path runs before direct P2P, SFU and peer-relay routes. - -The earlier delayed self-echo report on `769de201f7cc` has a different boundary: -a viewer's voice is rendered by a separate voice application on the Host, -enters the system mix, and is captured and sent back. It is source-content -leakage, not ordinary microphone acoustic echo. AEC, noise suppression, or -post-mix Web Audio cannot recover the originating process and must not be -presented as a fix. - -On 2026-08-21 the production picker was user-verified to keep tab and window -audio scoped as intended in that browser environment. Current source requests -the source-appropriate window and full-display audio choices. This is one -environment observation, not a cross-browser source guarantee. - -## Voice-Processing Boundary - -Extension `tabCapture`/`desktopCapture` tests do not establish speech-processing -defaults for the Web Screen Capture API. Chromium M142 classifies ordinary web -`getDisplayMedia()` as -`kOther`, not `kExtensionScreenShare`. Without explicit constraints, that path -selects browser-decided echo cancellation and defaults noise suppression and -automatic gain control on. Its processed candidate also defaults to one channel. - -This is real source processing, not just settings metadata: -`MakeForDisplayCapture()` creates a local WebRTC audio-processing module and -uses PeerConnection playout as the echo reference. Setting `contentHint` to -`music` after capture does not rewrite those source properties; current Blink's -WebRTC audio sink has no matching handler, and libwebrtc's Opus encoder retains -a TODO for content-hint use. - -The Chrome/Edge capture request therefore supplies bare `false` constraints for -echo cancellation, noise suppression, automatic gain, and voice isolation, plus -`channelCount: { ideal: 2 }`. These are post-selection preferences, not source -picker restrictions or cross-browser guarantees. The declared Host baseline -accepts them; a browser may ignore unsupported dictionary members. No `exact`, -`min`, `advanced`, or fixed sample rate is requested, so this change does not -turn optional screen audio into a hard capture gate. - -## Standard API Boundary - -| Concern | Standard request or control | Honest readback | What it does not prove | Current product action | -| --- | --- | --- | --- | --- | -| Audio presence | A `MediaTrackConstraints` audio dictionary expresses interest | `stream.getAudioTracks().length` proves only whether a track was returned | The browser may still return video only; a track does not identify system, window, tab, or selected-game audio | Keep the existing presence check and visible no-audio warning | -| Audio source choice | `windowAudio: "window"` asks for window audio and `systemAudio: "include"` offers system audio for monitor surfaces; a user agent may ignore either hint | There is no standard audio-source category readback corresponding to those hints | The selected audio scope, per-application isolation, or cross-browser availability | Let the picker expose the source-appropriate option without adding an inferred source label | -| Content intent | `track.contentHint = "music"` records the source intent | The assigned hint can be read back | A codec, bitrate, stereo mode, or evidence that current Chromium changed Opus encoding | Keep it as metadata only; do not use it as a quality acceptance signal | -| Voice-app exclusion | `restrictOwnAudio` concerns audio produced by the document that invoked capture; `suppressLocalAudioPlayback` concerns local playback of a captured browser surface | The app may observe whether a returned track exists, not which OS processes it contains | Excluding Discord, KOOK, WeChat, notifications, or any other independent process from system audio | Web keeps video-only available and warns that system audio may include calls/notifications; it does not claim isolation | -| Source processing | Current Chromium accepts EC/NS/AGC/voice-isolation constraints on web display audio | Corresponding `track.getSettings()` fields can confirm values when exposed | Portable support, or that a missing field means false | Request all four off for movie/game audio and verify the target Host settings | -| Capture channels | Current Chromium accepts `channelCount: { ideal: 2 }`; the Screen Capture specification does not make it portable | `track.getSettings().channelCount` may describe the returned track when exposed | That every source/browser can supply stereo, or that RTP sends stereo | Prefer two without using `exact`; keep negotiated Opus truth separate | -| Capture sample rate | The Screen Capture specification does not list generic `sampleRate` as applicable to display audio | `track.getSettings().sampleRate` may be present in an implementation | The Opus mode, RTP clock semantics, receiver output rate, or end-to-end fidelity | Observe only in a future diagnostic; absent means unknown | -| RTP send bitrate | `RTCRtpSender.setParameters()` can set `encodings[].maxBitrate` for audio; Opus `maxaveragebitrate` advertises the receiver's maximum | A following `getParameters()` can show the applied sender ceiling; negotiated fmtp shows the receiver maximum; outbound byte deltas show actual traffic | A minimum, audible improvement, or moment-to-moment rate; congestion may keep traffic lower | Apply the selected 64/128/256 kbps sender ceiling, advertise one 256 kbps receive maximum, and retain actual bitrate as the acceptance fact | -| Codec | The user agent chooses among negotiated send codecs unless a separately negotiated codec selection is available | `RTCCodecStats.mimeType` and `sdpFmtpLine` identify the codec and negotiated format parameters in use | That Opus was selected before stats exist, or that negotiated preferences describe actual content | Label only observed negotiated data; do not force a codec | -| Stereo | No stable sender parameter controls Opus stereo; RFC 7587 defines `stereo` as the receiver's one-way preference | Capture settings and negotiated codec/fmtp data can be inspected separately | Capture channel count and `opus/48000/2` do not prove encoded stereo | Use one bounded Opus answer transform on the declared Chrome/Edge baseline; no user control or inferred badge | -| DTX | The current WebRTC `RTCRtpEncodingParameters` dictionary has no `dtx` member; RFC 7587 defaults absent `usedtx` to `0` | No portable standard sender readback proves DTX operation | A deprecated browser field or SDK option is not a cross-browser contract | Keep peer default-off and set LiveKit `dtx: false`; no user control | -| FEC | There is no direct, portable audio FEC on/off sender parameter | Negotiated codec/fmtp data and inbound `fecPacketsReceived`/`fecBytesReceived` may provide evidence after use | Negotiation does not prove recovery occurred; zero counters do not prove FEC was disabled | No control; retain stats only as future diagnostic evidence | - -The WebRTC specification defines audio `maxBitrate` as a maximum and allows -other limits to constrain the sender further. A retained Chrome 151 loopback -showed only that the browser applied `maxBitrate` and changed its encoder budget -for that synthetic fixture; the ordinary PC, short interval and synthetic -source are not calibration evidence. RFC 7587's full-band stereo music range -and the pinned LiveKit preset justify the product value. Audible quality, -actual traffic and congestion behavior remain runtime observations. - -## Capture And Browser Compatibility - -The Screen Capture Working Draft says constraints are applied only after the -user chooses a surface and that `getUserMedia()` constraints do not apply to -display tracks unless this specification lists them. The only portable audio -properties it currently lists are `restrictOwnAudio` and -`suppressLocalAudioPlayback`; neither controls fidelity. Chromium's processing -and channel behavior is therefore an implementation contract for the declared -Chrome/Edge Host baseline. The specification also explicitly permits a browser -to return no audio despite an audio request and rejects `advanced` or `min`/ -`exact` display constraints; this request uses none of them. - -MDN browser-compatibility data on the access date reports: - -- display-audio capture in Chromium from Chrome 74, with whole-system audio on - Windows and ChromeOS but tab-only audio on Linux and macOS; -- no display-audio capture support in Firefox or Safari; -- `systemAudio` in desktop Chromium from Chrome 105, but not Firefox or Safari; -- `windowAudio` is listed as partial in Chrome 141: `"exclude"` and `"system"` - are documented while `"window"` is not yet represented as supported in BCD; - current Chrome deployments may differ, so runtime/user evidence stays local; - and -- `MediaTrackSettings.channelCount` as non-Baseline because it is missing in - widely used browsers. - -These are an implementation snapshot, not permanent product guarantees. -Chromium's own capture documentation also separates Windows/ChromeOS system -loopback capture from tab capture. It publishes no stable Web contract for a -fixed display-audio sample rate or channel count. - -`encodings[].maxBitrate` has much broader compatibility: MDN data records -Chrome 69, Firefox 46, and Safari 11. The old `encodings[].dtx` field is absent -from Chrome and Firefox, present only as a deprecated non-standard Safari -feature, and absent from the current WebRTC IDL. - -## Opus Signaling Is Not Source Truth - -WebRTC endpoints must implement Opus, but negotiation may select another -mandatory audio codec. The application must wait for the in-use codec stats -before calling a stream Opus. - -RFC 7587 requires an Opus SDP `a=rtpmap` clock rate of 48000 and channel count -of 2 for all Opus sessions. It separately explains that actual Opus media may -use another internal sampling rate. Therefore `audio/opus`, `clockRate=48000`, -and `channels=2` in codec stats do not establish a 48 kHz stereo capture or -stereo payload. - -Opus `maxaveragebitrate`, `stereo`, `useinbandfec`, and `usedtx` are fmtp -offer/answer parameters. `stereo` is a receive preference, while -`sprop-stereo` is only a sender hint and explicitly not a guarantee. A browser -API that merely exposes the negotiated `sdpFmtpLine` can report those facts, -but does not provide a portable setter or prove the encoder's moment-to-moment -behavior. - -Current libwebrtc source calculates the default full-band Opus bitrate as -32 kbps times the negotiated channel count. It derives two encoded channels -only from `stereo=1`, so stereo without `maxaveragebitrate` starts from a -64 kbps codec bitrate. `RTCRtpSender.maxBitrate=128000` only caps the allocator; -it does not raise that codec configuration. When present, libwebrtc maps the -negotiated `maxaveragebitrate` value into the Opus encoder bitrate config before -normal congestion adaptation. This is implementation evidence for the current -Chrome/Edge baseline, while the RFC still defines the parameter as a receiver -maximum rather than a minimum or continuous bitrate guarantee. - -The poor movie/music report against production `6ccb516` began under a -historical mono negotiation contract, but it remained after a later 128-only -source stage added peer `stereo=1;maxaveragebitrate=128000` and SFU -`forceStereo: true`, 128 kbps, DTX off, and RED retained. Current Share advanced -settings provide 64/128/256 kbps sender ceilings with 128 kbps as the default; -those encoder settings still cannot restore PCM -already changed or collapsed to mono by the Host capture processor. Loss, -jitter and concealment remain separate diagnostic facts, but direct SFU -reproduction makes source processing the first reversible correction. - -## Reference Implementations - -No code was copied. All repositories were inspected at pinned commits on -2026-08-19 and their cited source pages were rechecked on 2026-08-21. - -- LiveKit client-sdk-js 2.22.0, Apache-2.0, commit - [`0a2110d`](https://github.com/livekit/client-sdk-js/tree/0a2110d39904a06722a0c4d1ddbb9390bb06ad4d): - it treats - [`getSettings().channelCount` or requested constraints](https://github.com/livekit/client-sdk-js/blob/0a2110d39904a06722a0c4d1ddbb9390bb06ad4d/src/room/participant/LocalParticipant.ts#L899-L920) - as a stereo-input heuristic and populates standard - `RTCRtpEncodingParameters.maxBitrate`, but stereo/DTX/RED also travel in - LiveKit signaling. Its transport then - [munges Opus bitrate fmtp](https://github.com/livekit/client-sdk-js/blob/0a2110d39904a06722a0c4d1ddbb9390bb06ad4d/src/room/PCTransport.ts#L269-L333) - in SDP. Its room defaults are music 48 kbps, DTX on, RED on and - `forceStereo: false`; its named presets include stereo music at 64 kbps and - high-quality stereo music at 128 kbps. Per-track options merge after those - defaults. Its stereo-only `??=` fallback disables DTX/RED only if still - undefined, which the pinned room defaults are not. Screener's historical - pre-stereo source overrode DTX but not `forceStereo` or RED, yielding mono, - DTX off and RED on. -- lib-jitsi-meet, Apache-2.0, commit - [`63a04ec`](https://github.com/jitsi/lib-jitsi-meet/tree/63a04ecabd972ea75e877f9ba12086c13cb68210): - [`mungeOpus()`](https://github.com/jitsi/lib-jitsi-meet/blob/63a04ecabd972ea75e877f9ba12086c13cb68210/modules/RTC/TPCUtils.ts#L871-L929) - writes `stereo`, `sprop-stereo`, `maxaveragebitrate`, and `usedtx` into SDP, - including a Firefox-specific DTX exception. -- simple-peer, MIT, commit - [`f1a492d`](https://github.com/feross/simple-peer/tree/f1a492d1999ce727fa87193ebdea20ac89c1fc6d): - it has no higher-level audio quality contract and exposes only an advanced - generic - [`sdpTransform`](https://github.com/feross/simple-peer/blob/f1a492d1999ce727fa87193ebdea20ac89c1fc6d/index.js#L608-L664) - hook for SDP changes. - -The Apache and MIT implementations support the same conclusion: one narrow, -structured Opus fmtp transform is established practice, but it is still an -application-owned negotiation contract rather than a portable sender setter. -`sdp-transform` 2.15.0 is MIT-licensed and already arrives transitively through -LiveKit. Runtime code that imports it must declare it directly, together with -its TypeScript types, rather than depend on LiveKit's dependency graph. -As supplementary practice evidence, historical -[LiveKit server issue 970](https://github.com/livekit/livekit/issues/970) and -the 2026 [Rust SDK issue 1018](https://github.com/livekit/rust-sdks/issues/1018) -both report stereo sources arriving as mono when the product/SDK did not signal -stereo. Issue reports are not treated as normative behavior or as proof of -Screener's runtime state; pinned source and local stats remain authoritative. - -Discord and Oopz have a narrower evidence boundary: - -- Discord's official Go Live architecture says its native client captures the - selected process and child-process audio with OS-specific APIs and sends - audio and video in separate RTP packets. Discord's public Voice protocol - requires Opus at the protocol boundary and distinguishes a `Soundshare` - speaking bit, but neither source publishes the exact negotiated/runtime - audio parameters for each consumer Go Live session. This is useful native - product evidence, not a preset Screener can copy or an excuse to infer codec - state without stats. -- Oopz's official help separates program-audio screen sharing from voice - channels: it documents Windows program-audio capture, while separate voice - articles expose channel quality/latency choices and microphone AI noise - suppression. The retained read-only - [Oopz 0.87.425 package inspection](./native-shared-encode-sender.md) found a - native Agora RTC library, program/system screen-audio switches, and Agora - profile symbols including high-quality stereo music and game-streaming - scenarios. This proves only separate native capture/voice/profile surfaces. - Neither the package nor public help proves the active screen-share codec, - bitrate, stereo, DTX, FEC or Agora profile; those values remain unknown. - -## Honest Diagnostics Contract - -Audio diagnostics keep four layers separate: - -1. **Capture:** track absent/present, plus `channelCount` and `sampleRate` only - when the browser returns numeric values. -2. **Requested/applied:** an RTP ceiling only if Screener actually calls - `setParameters()`, followed by immediate `getParameters()` readback. -3. **Negotiated:** current local details follow the unique audio RTP object's - `codecId` to a same-transport `RTCCodecStats` and display MIME type, clock - rate, channels and bounded fmtp as negotiated fields. Raw SDP/stats/fmtp are - not uploaded or persisted. -4. **Observed:** current local details derive interval audio bitrate from byte - deltas and interval loss from `lostDelta / (receivedDelta + lostDelta)`, and - display reported jitter. For the unique current inbound screen video/audio - pair, they also display audio-minus-video `estimatedPlayoutTimestamp`, - interval average jitter-buffer delay as - `delta(jitterBufferDelay) / delta(jitterBufferEmittedCount)`, audio concealed - samples as `delta(concealedSamples) / delta(totalSamplesReceived)`, and the - interval `concealmentEvents` count. RTP stats id, SSRC or track-identifier - changes rebase that media kind; a counter reset rebases the affected - interval. The first sample, negative delta, zero denominator, ambiguous RTP - object or absent field is unknown. These local-only fields are not added to - Viewer quality-evidence signaling or persistence. Browser availability and - actual route-switch synchronization remain target-device evidence; FEC - counters remain an acceptance-matrix follow-up. - -Never substitute `48000`, `2`, `false`, or `0` for an unavailable field. - -## Minimal Route-Consistent Runtime Slice - -The route-consistent runtime covers capture and negotiation without replacing -browser congestion control: - -1. Request display audio with echo cancellation, noise suppression, automatic - gain, and voice isolation disabled plus ideal two-channel capture. Keep - `contentHint = "music"` as metadata and one audio track. Apply the selected - 64/128/256 kbps sender ceiling, defaulting missing/legacy state to 128 kbps. - Do not force a sample rate or add Web Audio mixing, - resampling, a second representation or a new rate controller. -2. On every Web Viewer answer, including ICE restart/rebuild answers, parse SDP - with direct dependencies on MIT-licensed `sdp-transform` 2.15.0 and its - TypeScript declarations. Select the single non-rejected audio media section, - resolve its Opus payload from structured `rtp` entries, and idempotently - upsert `stereo=1` plus `maxaveragebitrate=256000` in that payload's fmtp before both - `setLocalDescription()` and signaling the same description. Preserve every - other media section, codec, fmtp key and session attribute. Empty SDP, no - unambiguous active audio/Opus payload, multiple active audio sections, - malformed fmtp, or parser/write failure returns the original answer object - and SDP unchanged. This keeps a browser-generated mono/video answer usable; - no broad regex or malformed partial answer is allowed. RFC 7587 makes stereo - an optional receive preference, while W3C guarantees the unmodified - `createAnswer()` remains usable by `setLocalDescription()`. -3. Do not add `sprop-stereo`: RFC 7587 defines it as a sender-likelihood hint, - not the receiver preference that permits the remote encoder to send stereo, - and offer/answer parameters are orthogonal. Do not add `usedtx=0`; absent - `usedtx` already means off. `maxaveragebitrate=256000` is the receiver maximum - covering all three sender ceilings, not a minimum or current target; stock - congestion control can still lower actual traffic. Keep the browser's - existing `useinbandfec` negotiation untouched. -4. For SFU publication map 64 kbps to LiveKit 2.22.0 - `AudioPresets.musicStereo`, 128 kbps to `musicHighQualityStereo`, and 256 kbps - to an explicit preset; retain `forceStereo: true` and `dtx: false`. - Do not override the pinned room default `red: true`. RED and Opus in-band FEC - are distinct repair mechanisms; retaining one neither proves, replaces nor - disables the other, and no FEC switch is exposed or reimplemented. - -Applying the receive preference in every browser answer covers direct P2P and -browser-relay children. SFU uses the equivalent SDK contract. -An SFU-only change is not acceptable: moving between peer and SFU routes would -otherwise change channel behavior. Native/Pion sender routes retain their own -explicit encoded-track contract and need a focused compatibility check before -claiming the same result. - -At the default ceiling, audio contributes at most 128 kbps RTP payload per -active outbound edge, or 256 kbps for the configured two-slot endpoint example. -The endpoint cap itself follows the current route policy. The -explicit highest preset doubles those bounds to 256/512 kbps, excluding -RTP/SRTP/UDP/IP overhead. A Host SFU publication contributes one such ingress; -each root/peer forward still sends its own copy. This is small beside the -current 3--8 Mbps video presets, but stereo can use more encoder work and -bandwidth than mono. No second track or representation is added, and browser -relays still make no shared-encoder promise. - -Preserve the existing persistent Viewer `MediaStream`/media element and add or -remove its audio/video tracks in place. Introducing a separate audio element or -Web Audio graph would create a new clock/buffering boundary and is outside this -fix; the stable stream already deployed is the A/V synchronization baseline. -The local receiver diagnostic observes only `getStats()` on that existing -route. It does not set `jitterBufferTarget`, create a clock, modify playout, add -a wire field, or claim that a missing browser statistic is zero. - -The implementation budget is deliberately bounded to dependency -manifest/lockfile entries, one SDP helper, `ViewerPeer` answer wiring, SFU -publisher options, focused unit/integration tests and the owning docs. Parser -tests cover CRLF serialization, an existing/missing fmtp line, exact stereo and -bitrate replacement, unknown fmtp preservation, idempotence, payload-number -lookup, rejected/non-audio sections, and identity-preserving fallback for empty -SDP, no/ambiguous Opus, duplicate targets, and parser exceptions. Peer tests -prove the exact selected answer is both applied and -signaled on initial/restart/rebuild paths; SFU tests assert the three explicit -options and absence of a RED override. One Chrome/Edge left/right fixture checks -channel correctness and a P2P/SFU route switch checks the persistent stream; -neither is a performance benchmark or parameter-calibration gate. - -## Native Windows Process-Audio Candidate - -The smallest honest native path is Windows WASAPI application loopback through -`ActivateAudioInterfaceAsync` and `AUDIOCLIENT_PROCESS_LOOPBACK_PARAMS` with -`INCLUDE_TARGET_PROCESS_TREE`. After the user selects a game window, the sender -resolves its PID locally and captures that process plus children. The API's -minimum build 20348 is not available on ordinary Windows 10 22H2 (19045), so -the current product candidate covers Windows 11 only. Windows 10 game-only -audio remains unresolved or explicitly unsupported until another bounded path -passes. - -Default native behavior is game-only. Whole-system audio requires an explicit -choice. No render stream, process exit, or capture failure produces a visible -silent/failed state and never widens capture to the whole system. Excluding one -known voice process is not a sufficient default because other calls, -notifications, or browser audio could still leak. Protected content may be -absent and an exclusive/no-render path may yield silence rather than a -classifiable error; the sender reports silence or unknown and never bypasses -protection. PID, path, window title, device identity, and raw audio remain local -and are not retained or uploaded. - -### P1 Audio-Only Wiring Slice - -The first native implementation is intentionally an audio-only input and is -estimated at 500--800 new lines. It keeps the current Web video capture and -codec path, while a Windows 11 helper supplies process-tree PCM through the -existing authenticated local bridge. Chrome/Edge's already-required WebCodecs -surface encodes that PCM with `AudioEncoder` (`opus`, 48 kHz, two channels, -20 ms), so the helper does not add a native Opus dependency or a second package -toolchain. The Go sender packetizes the returned Opus access units once into a -shared `audio/opus` RTP track before adding it to each native PeerConnection. - -The local envelope carries only media kind, flags, QPC-derived timestamp, -duration, and bounded payload; no PID or source label crosses the bridge. WGC -video remains a fixture/next adapter in this slice. Its -`SystemRelativeTime`, like WASAPI `GetBuffer`'s `pu64QPCPosition`, is a 100-ns -QPC value and is retained as the future A/V synchronization origin. Audio -timestamps are converted to the 48 kHz RTP clock and remain monotonic; packet -duration comes from the captured frame count rather than arrival time. - -This is an opt-in P1 candidate, not a default or production switch. Windows 10, -picker/activation denial, an exited or inaccessible target, no render stream, -protected-content silence, and WASAPI device/service errors report -`unavailable`/`silent` and stop or ask. The implementation never widens to -whole-system loopback. The first acceptance smoke is one direct Viewer with a -known target tone/visual marker and independent voice/notification markers; -SFU/UDP, a second Viewer, endurance, and native WGC video replacement remain -later gates. - -### Retained P1 Result (2026-08-21) - -The source-complete Windows helper emits process-tree-only 48 kHz stereo s16 in -20-ms chunks. Chrome 151 encodes the chunks once with `AudioEncoder`/Opus, and -Go packetizes the same encoded stream once into a shared Pion track. The UI -receives an opaque memory-only target ID. PID and the enumeration-time process -creation token are revalidated locally and never enter public signaling, a URL, -storage, or diagnostics. - -One bounded run played 440 Hz in the selected target and 880 Hz in an -independent process. Across 100 chunks, amplitudes were 0.240225 and 0.0000598 -(4017.8x); no system-mix fallback was used. One direct Viewer then received one -audio track and 495 inbound Opus packets while video decoded/rendered 296 -frames. There was no fatal or encoder error. The sole failed assertion was the -existing generic Pion two-second outbound snapshot, which also missed fresh -deltas in earlier video loopbacks. This is functional evidence, not packaging, -real-game sync, second-Viewer, SFU/UDP, or endurance evidence. - -Run one bounded matrix rather than a full route Cartesian product against -the exact production release and matching runtime source at execution time on -Windows Chrome/Edge for tab/window/monitor, -audio selected/unselected, and a simultaneous voice call; then the native -candidate on current Windows 11 with game parent and child audio, an independent -voice process, notifications, no render stream, and process restart. Windows 10 -records the explicit unsupported/unresolved result rather than a fake fallback. -Direct is the primary route; one browser-relay and one SFU-subscription audio -check cover route preservation. Ordinary peer ICE remains STUN-only and current -SFU media remains UDP-only; this audio matrix does not add a transport path. -Correlate capture settings, negotiated codec/derived fmtp, -actual outbound/inbound bitrate, loss, jitter, concealment, jitter buffer, and -A/V playout timing using a distinguishable stereo fixture plus game/film audio. - -## UI And Voice Boundary - -Share advanced settings offers exactly 64/128/256 kbps and defaults to 128. -Current source applies the selected choice before sharing, to every newly -created P2P, browser-relay, or SFU sender, and through a serialized live -mutation; production deploys the same path. It reads -the latest desired profile into each sender and keeps endpoint-local applied -readback rather than claiming one room-wide applied commit. The setting remains -a sender ceiling, not a guaranteed or constant bitrate. Do not expose sample -rate, channel count, codec, DTX, RED, FEC, an arbitrary slider, or a second audio -adaptation loop. - -If microphone voice enters scope, treat it as a separate source/track with an -independent privacy, AEC, noise-suppression, gain and DTX design. Oopz's public -product surfaces support that separation but do not supply parameters that can -be copied into Screener's screen-media path. +# Browser Screen-Audio Quality + +- Research date: 2026-08-25 +- Scope: Browser game/movie screen audio over P2P, relay, and LiveKit SFU +- Status: current Browser contract implemented and deployed; heterogeneous + devices and real-game A/V synchronization remain open + +## Current Conclusion + +Screener requests display audio by default on initial capture and source switch. +If the browser, selected source, operating system, or user returns no audio +track, video sharing continues with an explicit warning. + +One screen-audio track uses `contentHint = "music"`. Capture requests ideal +stereo and disables echo cancellation, noise suppression, automatic gain, and +voice isolation where the browser supports those constraints. These are +preferences and readback fields, not portable fidelity guarantees. + +The only user-facing audio-quality choices are sender ceilings of 64, 128, and +256 kbps, with 128 kbps as the default. They update live on current P2P, +Browser-relay, and SFU senders and apply to future senders. Actual RTP traffic +can remain below a ceiling because of content and congestion, and includes +transport behavior not represented by the selected number. + +## Peer Contract + +Direct and Browser-relay Viewer answers use one structured `sdp-transform` +operation on the single active Opus audio section. It idempotently requests +`stereo=1` and `maxaveragebitrate=256000`, preserving every unrelated codec, +format parameter, media section, and session attribute. Missing, ambiguous, or +malformed targets return the browser-generated answer unchanged. + +The 256 kbps receive maximum permits every sender ceiling; it is not a current +target or minimum. Screener does not set `sprop-stereo`, rewrite FEC, force a +sample rate, or add broad string/regular-expression SDP mutation. + +Each audio sender reads fresh parameters, applies the selected `maxBitrate`, and +reads the result back. A rejected update leaves the previous sender state and +media alive. Without a remote applied acknowledgement, the Host does not claim +room-wide atomic convergence. + +## LiveKit Contract + +The SFU publisher maps the three ceilings to pinned LiveKit stereo music presets +or an explicit 256 kbps preset. Every SFU audio publication uses: + +- `forceStereo: true`; +- `dtx: false`; and +- `red: false`. + +RTP RED sends redundant copies of prior audio data to improve recovery from +short packet-loss bursts. It also materially increases payload traffic; the +earlier 128 kbps selection produced about 256 kbps on the measured SFU path. +Screener disables RED so the selected audio ceiling remains a useful publisher +budget. This trades away RED-specific loss resilience. Opus in-band FEC and +other WebRTC recovery remain separately negotiated; disabling RED neither proves +nor disables them. + +LiveKit reconnect reacquires the exact current publication and sender before +reapplying audio parameters. Unchanged publication/sender identity does not +reset stats or parameter state. Missing current ownership fails the SFU route +closed instead of mutating a stale sender. + +## Relay And Presentation + +A Browser relay keeps one persistent `MediaStream` and reconciles screen-audio +track addition, removal, and replacement into each current child sender. A +source switch does not create an independent audio clock or Web Audio graph. + +Viewer playback uses the browser's native media controls on the one persistent +video element. Local volume, mute, play/pause, and fullscreen do not enter +signaling, sender configuration, room state, or another participant. Autoplay +rejection remains a typed presentation state until the browser receives a user +gesture. + +Connection metrics select the exact current audio sender/receiver. When audio is +active, the UI reports actual interval traffic rather than a constant or +placeholder `1 kbps`. Track and sender replacement reset only the affected stats +identity, so stale asynchronous samples cannot overwrite the new route. + +## Evidence Boundary + +Chrome 151 production gates exercised direct Host children, a Browser-relay +child, and an SFU subscriber with generated screen audio. Video and decoded +audio energy advanced at all three 64/128/256 kbps ceilings and after source +replacement. Direct and Browser-relay traffic tracked the selected ceiling; +the then-enabled SFU RED path explained the approximately doubled SFU audio +payload. + +The same evidence verified that the active-audio display no longer reports the +old constant `1 kbps`. It did not establish perceptual music quality, all +capture-source combinations, weak-network loss recovery after RED is disabled, +or real-game SFU A/V synchronization. + +## Boundaries + +- No microphone/voice processing policy is inferred from screen audio. A future + microphone is a separate source and track. +- No user control for codec, stereo, channel count, sample rate, DTX, RED, FEC, + or arbitrary bitrate. +- No Web Audio mixing, resampling, second representation, or independent audio + rate controller. +- `opus/48000/2`, capture channel count, and configured ceilings do not prove + encoded stereo or end-to-end fidelity. +- Browser/system capture scope may include calls or notifications; the page + cannot promise per-process isolation from generic system audio. + +## Current Verification Gaps + +- Chrome/Edge tab, window, and system-audio source combinations on supported + Windows versions. +- Real game/music subjective quality and A/V synchronization over direct, + relay, and SFU paths. +- Lossy Wi-Fi/cellular behavior with RED disabled, including concealment and + Opus FEC counters where exposed. +- macOS, Android, and iOS capture/playback behavior. ## Primary Sources -- [W3C Screen Capture](https://www.w3.org/TR/screen-capture/) -- [W3C WebRTC](https://www.w3.org/TR/webrtc/) -- [W3C WebRTC Priority Control](https://www.w3.org/TR/webrtc-priority/) -- [W3C WebRTC Statistics](https://www.w3.org/TR/webrtc-stats/) -- [W3C Web Audio](https://www.w3.org/TR/webaudio-1.1/) -- [W3C WebCodecs Opus registration](https://www.w3.org/TR/webcodecs-opus-codec-registration/) -- [libwebrtc Opus encoder](https://webrtc.googlesource.com/src/+/refs/heads/main/modules/audio_coding/codecs/opus/audio_encoder_opus.cc) -- [libwebrtc media-track interface](https://webrtc.googlesource.com/src/+/refs/heads/main/api/media_stream_interface.h) -- [Chromium `MediaStreamTrack` content hint](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/third_party/blink/renderer/modules/mediastream/media_stream_track_impl.cc) -- [Chromium web display-audio constraint selection](https://github.com/chromium/chromium/blob/3620c35de32f20cfb11d0a616227c44750e31c67/third_party/blink/renderer/modules/mediastream/media_stream_constraints_util_audio.cc) -- [Chromium M142 display-audio default restoration](https://chromium.googlesource.com/chromium/src/+/b059fa325c6da901f1b0b6afd9e736d67f62960e) -- [Chromium display-capture audio processing](https://github.com/chromium/chromium/blob/0d07b03783490c156526384073fb5e97e7463e77/third_party/blink/renderer/modules/mediastream/media_stream_audio_processing_layout.cc) -- [libwebrtc stats collector](https://webrtc.googlesource.com/src/+/refs/heads/main/pc/rtc_stats_collector.cc) -- [LiveKit 2.22.0 track options](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/track/options.ts) -- [LiveKit 2.22.0 publish defaults](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/defaults.ts) -- [LiveKit 2.22.0 local publication](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/participant/LocalParticipant.ts) -- [LiveKit 2.22.0 package dependencies](https://github.com/livekit/client-sdk-js/blob/v2.22.0/package.json) -- [Jitsi Opus SDP transform, pinned commit](https://github.com/jitsi/lib-jitsi-meet/blob/63a04ecabd972ea75e877f9ba12086c13cb68210/modules/RTC/TPCUtils.ts) -- [`sdp-transform` 2.15.0](https://github.com/clux/sdp-transform/tree/v2.15.0) -- [RFC 7587: RTP Payload Format for Opus](https://www.rfc-editor.org/rfc/rfc7587.html) -- [RFC 7874: WebRTC Audio Codec and Processing Requirements](https://www.rfc-editor.org/rfc/rfc7874.html) -- [Discord Go Live architecture](https://discord.com/blog/how-it-all-goes-live-an-overview-of-discords-streaming-technology) -- [Discord Voice protocol](https://docs.discord.com/developers/topics/voice-connections) -- [Oopz program/screen audio help](https://help.oopz.cn/42f9/54e2) -- [Oopz voice quality/latency help](https://help.oopz.cn/42f9/3020) -- [Oopz microphone-noise help](https://help.oopz.cn/42f9/f5ae) -- [MDN `getDisplayMedia()`](https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getDisplayMedia) -- [MDN `MediaTrackSettings.channelCount`](https://developer.mozilla.org/en-US/docs/Web/API/MediaTrackSettings/channelCount) -- [MDN Browser Compatibility Data: `MediaDevices`](https://github.com/mdn/browser-compat-data/blob/main/api/MediaDevices.json) -- [MDN Browser Compatibility Data: `RTCRtpSender`](https://github.com/mdn/browser-compat-data/blob/main/api/RTCRtpSender.json) -- [Chrome screen-sharing controls](https://developer.chrome.com/docs/web-platform/screen-sharing-controls) -- [Chrome 141 window-audio release note](https://developer.chrome.com/release-notes/141) -- [Chromium media-capture architecture index](https://chromium.googlesource.com/chromium/src/+/HEAD/docs/media/capture/) -- [Microsoft Application Loopback sample](https://learn.microsoft.com/en-us/samples/microsoft/windows-classic-samples/applicationloopbackaudio-sample/) -- [Microsoft `ActivateAudioInterfaceAsync`](https://learn.microsoft.com/en-us/windows/win32/api/mmdeviceapi/nf-mmdeviceapi-activateaudiointerfaceasync) -- [Windows process-loopback parameters](https://learn.microsoft.com/en-us/windows/win32/api/audioclientactivationparams/ns-audioclientactivationparams-audioclient_process_loopback_params) -- [Microsoft Windows Graphics Capture](https://learn.microsoft.com/en-us/windows/apps/develop/media-authoring-processing/screen-capture) -- [Microsoft `IAudioCaptureClient::GetBuffer`](https://learn.microsoft.com/en-us/windows/win32/api/audioclient/nf-audioclient-iaudiocaptureclient-getbuffer) -- [Windows 10 release information](https://learn.microsoft.com/en-us/windows/release-health/release-information) -- [Microsoft WASAPI loopback recording](https://learn.microsoft.com/en-us/windows/win32/coreaudio/loopback-recording) +- [Screen Capture](https://www.w3.org/TR/screen-capture/) +- [Media Capture and Streams](https://www.w3.org/TR/mediacapture-streams/) +- [WebRTC](https://www.w3.org/TR/webrtc/) +- [RFC 7587: Opus RTP payload](https://www.rfc-editor.org/rfc/rfc7587.html) +- [RFC 2198: RTP redundant audio](https://www.rfc-editor.org/rfc/rfc2198.html) +- [LiveKit audio RED and media options](https://docs.livekit.io/transport/media/advanced/) +- [LiveKit client 2.22.0 track options](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/track/options.ts) +- [sdp-transform](https://github.com/clux/sdp-transform) diff --git a/docs/research/realtime-quality-adaptation.md b/docs/research/realtime-quality-adaptation.md index efc80ed..9a48c81 100644 --- a/docs/research/realtime-quality-adaptation.md +++ b/docs/research/realtime-quality-adaptation.md @@ -1,834 +1,167 @@ # Realtime Screen-Share Quality Adaptation - Research date: 2026-08-25 -- Scope: realtime game screen sharing in the browser -- Status: exact-production Browser gate completed; real-game and heterogeneous-device quality remain unverified +- Scope: Browser game-screen capture, encoding, P2P forwarding, and LiveKit SFU +- Status: current Browser policy accepted; real-game, weak-device, and + heterogeneous-network quality remain open -## Finding +## Current Conclusion -Screener should not optimize one quality dimension at the expense of the -others. `maintain-framerate` may preserve motion by reducing resolution until -game UI, maps, subtitles, and text become unreadable; `maintain-resolution` -may instead lower frame rate. Neither preference overrides congestion control. +Browser video is fixed VP8. Display video uses `contentHint = "motion"`, and +display audio uses `contentHint = "music"`. The three recommended profiles and +advanced settings are ceilings, not delivery guarantees. WebRTC owns direct and +peer congestion control; LiveKit owns SFU representations, Dynacast, subscriber +bandwidth estimation, and layer forwarding. -Production runs exact deployed application/runtime revision -`be53c4d5794d38d5d406c876309958d7117ee601`, release `be53c4d`. Current Browser source and production use -strict `screener-v12`, fixed VP8, no video `contentHint`, and no codec UI, -quality state, or wire field. They use `balanced` as the recommended profile -and advanced default; -`maintain-resolution` and `maintain-framerate` remain explicit choices. These -preferences leave actual degradation to the browser, so Screener observes -readback and stats rather than claiming a fixed quality outcome. +Screener does not define a resolution/FPS/bitrate ladder, custom SFU lower +representation, scene detector, quality score, parent probe, periodic +rebalancing loop, or manual layer selector. It supplies standard content intent, +the selected sender ceiling, and readback of what the browser accepted. -The inspected Chromium/libwebrtc source chain makes a screen-only shortcut -especially unsafe to assume for Screener. The JavaScript `motion` hint reaches -`VideoRtpSender` as `kFluid`, which explicitly clears `options.is_screencast`. -`WebRtcVideoSendStream` then configures the encoder as `kRealtimeVideo`, while -`VideoStreamEncoder` only reinterprets `BALANCED` as `MAINTAIN_RESOLUTION` when -its encoder is marked as screen share. The explicit `balanced` preference still -wins the earlier preference-selection branch, but the later screen-only remap -does not follow from this project path. Treat the effective tradeoff as an -implementation detail to verify, not a product guarantee. +The active topology remains availability-driven. A hard connection failure, +non-paused 15-second decoded-frame stall, parent departure, or capacity +invalidation may trigger the ADR-0005 recovery operation. Loss, RTT, jitter, +bitrate, resolution, FPS, freeze counters, codec, and limitation reason are +diagnostic and cannot prove that another parent would be better. -An older libwebrtc experiment could detect animated screen content under -`balanced` and cap it to 1280x720. Commit `1d7d0e6e2c50` deleted that experiment -on 2024-05-22; the commit says it had already been disabled for several years -and was not maintained. Current Chromium behavior must therefore not be -described as automatically detecting game motion and forcing a 720p cap. +## Browser VP8 Evidence -## 2026-08-25 Exact-Production VP8 Gate +Chrome 151 exact-production tests with real `getDisplayMedia()` showed that +direct and Browser-relay VP8 can sustain about 60 fps after the stock bandwidth +estimator warms up. The observed ramp took roughly 25 to 30 seconds; a configured +60 fps or bitrate remains a ceiling rather than a startup promise. -Chrome 151 exercised exact production `bf32859` with real `getDisplayMedia()`, a -continuously changing source, separate Browser processes, VP8/libvpx, and no -video content hint. The direct path stabilized near 59.5 encoded, received, -decoded, and rendered fps at 1904x928 and about 7.5 Mbps. A Browser relay kept -every stage near 59.4--59.7 fps; its outbound adapted to 1428x696 under the -available bandwidth without losing frame cadence. Both paths needed about -25--30 seconds for stock bandwidth-estimation ramp-up. This closes the reported -intrinsic VP8/no-hint localhost 60 fps shortfall; sender ceilings are not startup -or delivery guarantees, and no application start-bitrate hack follows. +One controlled no-hint/motion comparison showed that Chromium may preserve +motion by spatially downscaling when `motion` is set. That is the standard +content-hint tradeoff for games, not a minimum-resolution contract. Leaving the +hint unset can preserve screen-classified resolution while shifting pressure to +quantization or frame delivery, so the earlier no-hint improvement was not free +adaptation. Current policy follows the standard game-motion intent and leaves +the resulting tradeoff to the browser. -The exact SFU path isolated a different issue. With the then-deployed ordered `q,h` -publication, the 1904x928 `h` stream stabilized near 9.2 fps and 1.37 Mbps. With -`q` inactive from the first sender-parameter application, the otherwise -unchanged `h` stream reached about 23.3 fps and 3.32 Mbps. Source/capture stayed -near 60 fps, encode cost was about 4.46 ms per frame, packet loss and -retransmission were zero, and the selected pair reported about 4.81 Mbps -available outgoing bitrate. The always-active lower representation therefore -consumed the same Host-to-SFU congestion budget and materially reduced `HIGH`. -This establishes a real dual-encode cost but does not justify removing the lower -representation needed by constrained subscribers. The public SFU ingress -remained constrained, so this gate does not claim SFU 60 fps or real-game -performance. +The Host configures a sender before the first offer and reapplies the current +video profile after accepting an answer. The latter is necessary because +negotiation may replace or rewrite encoding parameters; it is not a periodic +controller and does not reset bandwidth estimation on a timer. -## 2026-08-25 Exact-Production Single-Representation Gate +## LiveKit SFU Evidence -Chrome 151 exercised exact production `1d87615` with a real dynamic tab capture. -The bounded gate made only the ordinary STUN peer attempt unusable so the existing -route operation reached its SFU suffix; Browser LiveKit publisher and subscriber -PCs retained empty external ICE-server lists and selected UDP. The publisher -reported one video sender encoding and one outbound video stats object, both -without RID. Over six seconds it encoded 180 frames and the subscriber decoded -and rendered 182 frames at 1822x1080, with byte counters advancing on both ends. -This verifies the deployed single-representation baseline. It does not prove -that the baseline serves constrained subscribers and does not close the later -native-adaptation decision. It also does not establish public SFU 60 fps, -real-game quality, heterogeneous networks, or endurance. +Pinned LiveKit client `2.22.0`, server `1.13.5`, and Chrome 151 established that +the SDK's default VP8 screen-share simulcast can expose original and lower +representations, and that server send-side BWE can select a lower representation +for a constrained subscriber while an unconstrained subscriber receives the +highest available representation. -## 2026-08-25 Pinned Native-Adaptation Gate +An earlier exact-production A/B also showed that forcing an always-active lower +encoding can consume the same Host-to-SFU congestion budget and reduce the +highest encoding. That evidence rejects Screener mutation of active layers; it +does not justify single HIGH, which abandons constrained subscribers. -LiveKit server `1.13.5`, JS client `2.22.0`, and Chrome 151 exercised a changing -VP8 screen-share publication with full- and half-resolution representations. -Manual maximum/lower subscriber selection delivered `1280x720` and `640x360` -at about 30 fps, and the same connection returned from the lower to the full -representation. AdaptiveStream selected full resolution for a large attached -element, half resolution for a small one, paused when hidden, and recovered when -visible. Without an attached element it received only the lower representation, -so it cannot govern a Viewer that may forward the stream to peer children. +Current source therefore: -Dynacast stopped representations with no subscribers after about five seconds. -Lower-only demand stopped the full representation, but maximum demand kept both -encodings active. It therefore avoids unused work but cannot remove VP8's -dual-encode cost while a healthy maximum-quality subscriber exists. +- creates one Host publication for all SFU subscribers; +- sets VP8, no backup codec, the selected HIGH ceiling, and degradation + preference; +- leaves `screenShareSimulcastLayers` unset so pinned LiveKit owns construction + and republish behavior; +- enables Dynacast and server send-side BWE; +- leaves AdaptiveStream disabled because a Viewer may relay the received track + to children; and +- requests no application-selected subscriber layer. -With LiveKit's default receiver-side BWE, an approximately 0.96 Mbps subscriber -remained on HIGH for about 21 seconds and ended at zero decoded fps. With -`congestion_control.use_send_side_bwe: true`, LiveKit's native stream allocator -selected LOW and by the third steady window delivered `640x360` at about 29 fps -and 88 KB/s. A fresh unshaped subscriber immediately received HIGH. Network -unshaping on the same PeerConnection was not isolated because the Chrome DevTools -condition was bound when that connection was created; manual and AdaptiveStream -same-connection upward transitions independently proved layer recovery. +LiveKit documents automatic bandwidth-based layer selection. It does not expose +an automatic policy that observes delivered FPS and spatially downshifts solely +to satisfy Screener's `maintain-framerate` preference. Adding such a controller +or fork remains outside the current product. -The result accepts pinned-default VP8 dual-representation simulcast, Dynacast, -and server send-side BWE. -Screener does not add a bandwidth estimator or layer controller. AdaptiveStream -stays disabled for current subscribers because every Viewer may become a relay. -The earlier dual-encode cost is handled through explicit Host share profiles, -not by silently abandoning low-bandwidth Viewers. +## Codec Decision -## 2026-08-25 Content Classification And Bounded Scaling Gate +VP8 is the only Browser media codec because it has the broadest current Browser +interoperability and produced stable software-encoding behavior in the measured +Chrome path. The page does not select a particular VP8 hardware or software +encoder. -Chrome `151.0.7922.174` exercised one real `getDisplayMedia()` track, one VP8 -P2P sender/receiver pair, `maintain-framerate`, and the same deterministic -moving 1904x928 source for 30-second windows. At an 8 Mbps ceiling, no video -hint held about 60 fps at 1904x928 and about 8 Mbps with no resolution changes. -`contentHint = "motion"` held 60 fps but the native quality scaler reduced the -track and sender to 242x118 and remained there through the steady window. -Applying an 854x480 minimum track constraint before the motion run did not form -an encoder floor: the steady sender was 428x208 at about 60 fps and 8 Mbps. +Controlled Windows Chrome evidence localized the poor H.264 result to Chromium's +rate-control path around some Media Foundation encoders rather than a portable +page setting. The same AMD MFT reached the requested cadence only under a +diagnostic browser field trial that a Web application cannot enable. OpenH264 +software was not a superior fallback. H.264 remains research evidence for a +later native sender, not a Browser option. -The result matches Chrome's exact libwebrtc pin: VP8 automatic resize is disabled -for a screencast and enabled only for a realtime, single-active-stream encoder. -The motion hint therefore restores native bandwidth/QP spatial scaling by -changing the whole content classification, not by enabling an independent -quality switch. The available standard track constraint does not bound that -internal encoder scaling. Restoring motion would recover multiple resolution -steps but can make game UI unreadable, so current source must not restore it as -a global or fluid-profile shortcut. +HEVC, AV1, custom WebCodecs pipelines, and application packetization do not +replace the browser WebRTC sender without a new capture, RTP/RTCP, feedback, +hardware, interoperability, and licensing design. No such path is accepted. -This gate proves the mechanism, not visual quality for every game. It also means -the no-hint result is not a free efficiency gain: it preserves screen encoding -semantics and resolution while shifting pressure into quantization or frame -drops. A future bounded P2P spatial controller is a separate product mechanism, -not a hidden consequence of `degradationPreference`. +## Host Cost Boundary -The same production session exposed the separate SFU ingress boundary. A -1440p60/8 Mbps publication advertised a same-FPS half-resolution 2 Mbps lower -representation. The SFU Viewer received the highest representation at -2400x1350, about 37 fps and 1.84 Mbps while its downlink reported 54 ms RTT and -zero loss; the simultaneous P2P Viewer received about 59 fps and 5 Mbps. A -server interval measured about 4.46 Mbps ingress and 2.63 Mbps egress, consistent -with the unforwarded lower representation consuming Host-to-SFU budget. Later, -the native allocator selected only LOW and an RTP-layer sample observed LOW -traffic with no HIGH traffic, so downstream layer selection itself was not -stuck. The accepted correction keeps that native selector and reduces only the -60 fps lower representation to at most 30 fps using the SDK's proportional -bitrate formula. +One Chrome 151 VP8 sample on a Ryzen 7 9700X used about 0.79 logical core for one +1904x928@30 sender and about 1.34 logical cores for two independent senders. The +process totals included capture, WebRTC, source rendering, and local decode, so +they are not a weak-device or real-game capacity claim. Browser peers retain one +independent sender and congestion controller per `RTCPeerConnection`; shared +encoding is not guaranteed. + +The accepted way to handle a constrained Host is the existing explicit share +profile. Screener does not silently remove a representation needed by a weak +Viewer or invent a hidden host-performance tier. + +## Page Lifecycle Boundary + +Host document visibility, captured-source visibility, capture production, +encoder throughput, transport, Viewer decode, and local presentation are +different variables. Current code pauses only the Host's muted local preview +when its page is hidden or unfocused; it does not stop capture or senders. + +Browser timers and frame callbacks may be delayed while hidden. A resumed Viewer +must rebaseline the existing decoded-stall wall clock before another no-progress +sample can invalidate the route, and a new current-generation composited frame +is authoritative evidence that stale `connecting` or `reconnecting` +presentation has ended. These rules make recovery lifecycle-safe; they are not +a keepalive mechanism. + +The Screen Capture specification permits a user agent to mute a captured surface +that becomes inaccessible, including a minimized captured window. A page cannot +override that platform boundary. Mobile background audio, video recovery, page +reclamation, and relay survival remain physical-device acceptance work. ## Route Quality Authority -Current-path WebRTC stats can diagnose loss, RTT, jitter, bitrate, resolution, -FPS, freeze, encode/decode work, and limitation reason, but they cannot prove the -counterfactual quality of an unconnected parent. RFC 8836 assigns real-time -congestion adaptation to the media transport. The Overcast, Narada, and NICE -overlay algorithms require active measurements, periodic reevaluation, -thresholds, or additional overlay state; those mechanisms conflict with -Screener's one event-driven reconciliation loop, one room-serial child -operation, and sticky healthy edges. - -Screener therefore does not compute a weighted route score, probe alternative -parents, or reparent a currently decoding edge for bitrate, resolution, FPS, -freeze ratio, RTT, jitter, loss, or limitation evidence. Only a hard -`failed/closed` connection or the existing non-paused decoded-frame stall makes -the exact edge invalid. The current wire carries no parent-wide quality evidence -or threshold-based route authority. Primary sources checked 2026-08-24: [RFC -8836](https://www.rfc-editor.org/rfc/rfc8836.html), -[Overcast](https://www.usenix.org/legacy/publications/library/proceedings/osdi2000/full_papers/jannotti/jannotti_html/index.html), -[Narada](https://www.cs.cmu.edu/~srini/papers/papers/2002-Chu-jsac/2002-Chu-jsac.pdf), -and [NICE](https://conferences.sigcomm.org/sigcomm/2002/papers/appmulti.pdf). - -## 2026-08-19 Then-Production Degradation Report - -The then-production release `769de201f7cc` was reported to become severely blurry -while the host repeatedly showed a sustained native -`qualityLimitationReason=bandwidth`, despite one capable viewer on the same LAN -and the UI showing direct P2P. This is a user report, not yet an instrumented -root cause. The probe is read-only, requires three consecutive windows, and -does not change media. Same-LAN/direct does not exclude Wi-Fi loss or queuing, -per-PeerConnection GCC, aggregate host-uplink contention, or browser bandwidth -underestimation; isolated encoder pressure would more commonly report `cpu`. - -Only a full Host refresh and re-share was reported to restore immediate -high-definition, fluid video; Viewer refresh did not. With a stable -session-storage `clientId` and reconnect inside the five-second grace, Viewer -reauth reuses its `peerId` and emits `peer-joined`, while `HostPage.startPeer` -retains an already-connected `HostPeer`; a new tab, storage failure, or expired -grace can instead allocate a new peer. The reported Viewer refresh therefore -did not create a new host PeerConnection/GCC generation. Host-page cleanup -disposes every peer, -stops capture, and creates all-new capture, sender, PC, and GCC generations. -This makes a stuck host sender/PC/GCC or shared-capture generation a priority -hypothesis, not a conclusion. A further report that quality may remain low -after the old viewer leaves and a new viewer joins must likewise be tested -against lifecycle evidence rather than assumed. - -Use this as the first A+B/C reproduction before any route change or dual-layer -publication: - -1. Reproduce one wired LAN/direct viewer first, then Wi-Fi, two/three viewers, - and SFU/UDP, using the same high-motion scene for 60 to 90 seconds. -2. In one window record A capture; B actual/target/available outbound bitrate, - dimensions/FPS, codec, encode cost, loss/RTT/retransmission, selected path, - limitation, and PC/track generations; and C inbound decode/drop/freeze. -3. First retain capture and every healthy peer while rebuilding only the - affected `HostPeer`/`connectionId`. Separately compare capture/track - replacement with PCs retained, then a full Host rebuild; do not combine the - interventions. -4. For viewer churn and stable-`clientId` reconnect, retain old/new - `peerId`/`connectionId`, `peer-left`-to-dispose time, active sender count - inside/outside the five-second grace, old snapshot removal, whether a new PC - actually exists, capture settings, and new-path BWE/limitation. - -A single-edge rebuild passes only if the affected viewer recovers, unaffected -viewers neither migrate nor interrupt, and the host-edge cap holds. Only proven -generation-specific hard failure or non-paused decoded-frame stall can justify -automatic one-edge recovery under the current route attempt. Do not add -periodic reconnect, blindly raise a ceiling, or -change the current routing behavior before this gate. - -Missing `scaleResolutionDownBy` is not itself a root cause for a single encoding -because its effective default is 1.0. Likewise, the current 3/5/8 Mbps values -are ceilings rather than targets. Raising them cannot repair CPU or bandwidth -limitation and should only follow evidence that the encoder is already pinned -to the ceiling while spare transport capacity remains. - -## Confounded Same-Setting Observation - -On 2026-08-21 one bounded local Chrome 151 run used exact source `b447ab6`, a -synthetic 1920x1080 at 60 fps capture, and one direct UDP Viewer. After the -initial connection had existed for about 22 seconds, the Host outbound and -Viewer inbound were both 1280x720 at 57 fps and about 7.03 Mbps. The sender -reported `qualityLimitationReason=bandwidth`, despite about 12.59 Mbps of -available outgoing bitrate. Reapplying the identical 1080p60 `balanced` setting -through the real Host control produced 1920x1080 at 55 fps and about 7.11 Mbps -after 5.2 seconds, with the limitation cleared. Both ends reported zero packet -loss and no Viewer freeze. - -The route revision, PeerConnection identity, RTP SSRC, RTP stats object, and -track identifier all remained unchanged. This rules out a new route, sender, -or capture generation as the source of that observed transition. It does not -prove visual quality, production behavior, or SFU behavior. Earlier samples in -the same run remained at 640x360 through roughly 20 seconds and reached 720p in -the two-second window immediately before the control action, so continued -stock BWE ramp-up remains a causal confound. No second browser run was made. -The Host control applies same-track capture constraints before updating sender -parameters, so this run also cannot isolate those two standard calls. - -Static review does not support the simpler claim that Chromium discards every -pre-negotiation `setParameters()` call. Current libwebrtc stores sender init -parameters while no SSRC exists and transfers them into the negotiated sender. -The observed transition followed the same-setting action, but this single run -cannot attribute it to sender lifecycle or the reapply itself and therefore -does not support an answer-time whole-profile workaround. Current source keeps -the pre-offer sender configuration and serialized explicit source/profile -updates, but an accepted answer itself does not rewrite the profile. A future -workaround requires a controlled, real-capture reproduction that isolates one -intervention from stock BWE ramp-up. - -This observation alone did not justify an SFU change because stock BWE ramp-up -was not isolated. The later exact-production gate above independently measured -the SFU publication and rejected its always-active lower representation. -LiveKit's pinned client contains an SDP start-bitrate mitigation for initial -video blur, but Screener does not add application SDP bitrate hacks. - -The same release was also reported to reduce game-stream frame rate and -consume noticeable Host resources. That report applies only to -`769de201f7cc`, not to current production or automatically to the current -diagnostics. -The Web -sender creates one independent `RTCRtpSender` per viewer and has no cross-PC -shared-encoder guarantee; its muted local preview creates no media edge or -server traffic but may still consume compositor/GPU work. Compare the exact -release and exact current source under one fixture, with preview on/off as a separate -binary intervention. - -Desktop background diagnosis must also separate three independent variables: -Host document foreground/occluded/minimized state, captured source -foreground/occluded/minimized state, and local preview play/pause. Record -document visibility/focus, capture-track `muted`/`readyState`, actual codec, -capture/outbound/inbound FPS and bitrate, resolution, frames, encode time, and -limitation reason. The Screen Capture specification does not expose a page -keepalive contract; Screen Wake Lock is released when its document is no longer -visible, and browser page lifecycle may freeze JavaScript. Do not add fake -activity or keepalive timers. If the capture source or browser/OS suppresses -frames while minimized, record that standard capability boundary before any -packaged Host work. Primary sources checked 2026-08-24: [Screen -Capture](https://www.w3.org/TR/screen-capture/), [Screen Wake -Lock](https://www.w3.org/TR/screen-wake-lock/), [Page Lifecycle -API](https://developer.chrome.com/docs/web-platform/page-lifecycle-api), and -[Chromium desktop capture -implementation](https://chromium.googlesource.com/chromium/src/+/master/content/browser/media/capture/desktop_capture_device.cc). - -## SFU Profile Lifecycle And Publisher Evidence - -A 2026-08-21 report says that selecting fluid preference on an SFU path could -retain low received FPS without reducing the visible resolution. This is not -proof that the preference was ignored: `maintain-framerate` is a degradation -tradeoff rather than an FPS target. It is also not evidence of SFU temporal -downlayering: the then-deployed v10 Browser defaulted to VP8 but let the Host -select another value before starting the share. Pinned LiveKit server 1.13.5 installs a temporal selector -for VP8, but its H.264/H.265 path installs only the simulcast spatial selector. -With H.264 selected, a `HIGH` ceiling may therefore let BWE choose the -lower-resolution `q` representation, not a lower temporal layer at the same -resolution. When resolution remains stable, classify the case in A/B/C order: -capture FPS, Host publisher outbound FPS and limitation, then Viewer inbound FPS -and decode. - -Pinned LiveKit client 2.22.0 keeps three relevant pieces of state. Its public -`LocalVideoTrack.setDegradationPreference()` updates the saved preference used -when a sender is installed; `LocalVideoTrack.publishOptions` drives encoding -recomputation after a track restart; and `LocalTrackPublication.options` is the -input to `republishAllTracks()`. Current source keeps those three owners aligned -through the existing publisher operation queue and rollback: call the SDK -preference API to update its saved state, configure/read back the exact single -sender as the final write, then assign one merged option object to both retained -locations. Failure reapplies the previous profile; generation loss cannot -retain the result as current publisher state. This changes no capture -constraint, codec, representation, subscriber layer or route policy and does -not explain an immediate same-publication report. - -Host SFU publisher A+B remains a separate observability slice. Current source -implements one two-second, publication-generation-bound local sampler owned by -`SfuPublisher`: merge its video/audio `LocalTrack` reports, reuse the existing -strict stats parser and accumulator, correlate capture settings from the owned -video track, and emit only while the same publication is active. Activation, -replacement, deactivation and disconnect reset its identity and interval -baseline. `HostSfuRoute` may expose that local snapshot to one Host-only -publisher row; it must not duplicate the shared Host-to-SFU ingress inside each -SFU Viewer card. Viewer inbound remains the per-Viewer C signal. This needs no -wire, server telemetry, global score, selector or new UI framework. Production -deploys this publisher view; its target-browser fields and values still need -physical evidence. - -## Browser Codec And Content-Hint Evidence - -The accepted Browser contract is fixed VP8 with no video `contentHint`; returned -audio keeps `contentHint = "music"`. Direct and browser-relay offers contain VP8 -as their only video media codec, while RTX/RED/FEC may remain repair formats; -SFU publication explicitly uses VP8 with no backup codec. The UI and quality -wire expose no codec choice. Codec/profile/encoder stats remain diagnostic and -do not authorize automatic switching, route changes, or another controller. -Production runs exact deployed application/runtime revision -`be53c4d5794d38d5d406c876309958d7117ee601`, release `be53c4d`. Current source and production implement -this strict `screener-v12` contract. - -Chromium maps video `contentHint = "motion"` to libwebrtc `kFluid`, and -libwebrtc clears `is_screencast` for that mode, replacing the display-capture -screen classification with realtime-camera semantics. Fresh -Chrome 151 loopback probes on 2026-08-24 used real dynamic tab -`getDisplayMedia()`, `balanced`, ten seconds of warm-up, and a fifteen-second -sample: - -| Requested path | Ceiling | Actual encoder | Encoded FPS | Encode time/frame | Bitrate | End resolution | Limitation | -| --- | ---: | --- | ---: | ---: | ---: | --- | --- | -| VP8 + `motion` | 5 Mbps | libvpx software | 14.93 | 2.85 ms | 3.663 Mbps | 428x208 | `bandwidth` | -| VP8 + no hint | 5 Mbps | libvpx software | 29.73 | 6.62 ms | 4.944 Mbps | 1904x928 | `none` | -| H.264 + no hint | 8 Mbps | AMD Media Foundation | 14.18 | 14.24 ms | 8.214 Mbps | 1904x928 | `none` | -| H.264 + no hint + diagnostic Desktop SW BRC | 5 Mbps | AMD Media Foundation | 29.93 | 8.17 ms | 3.650 Mbps | 1904x928 | `none` | -| H.264 + no hint + hardware encode disabled | 5 Mbps | OpenH264 software | 10.47 | 13.08 ms | 4.832 Mbps | 1904x928 | `none` | - -The VP8 result establishes that `motion` was harmful independently of the H.264 -issue. VP8 reported trusted rate control, so libwebrtc's outer frame dropper was -off and libvpx owned its CBR behavior. In the ordinary AMD H.264 path, -libwebrtc's outer rate limiter dropped hundreds of frames; raising the ceiling -through 8/10/12 Mbps or selecting `maintain-framerate` did not restore 30 fps. -The diagnostic SW-BRC run made the same AMD MFT report trusted rate control and -reach 29.93 fps, so the MFT itself is not a fixed 12--15 fps throughput limit. -That browser-process experiment bypasses a Chromium AMD workaround and is not a -Web product API; a page cannot enable the field trial or select a GPU/MFT. - -A separate no-hint VP8 CPU sample used a Ryzen 7 9700X, Chrome 151, one dynamic -1904x928@30 capture, 5 Mbps per sender, and loopback receivers. One sender held -29.86 fps and the isolated Chrome process tree consumed 21.14 CPU-seconds over -26.65 seconds, or 0.793 logical core / 4.96% of the 16-thread machine. Two -independent senders both held 29.93 fps and used 35.88 CPU-seconds over 26.70 -seconds, or 1.344 logical cores / 8.40%. These totals include source rendering, -capture, WebRTC, and local decode, so they are not pure encoder cost. They show -that software VP8 is workable on this machine, not that it is cheap on a weaker -CPU, mobile device, or beside a real game. - -Chromium's generic WebRTC factory can wrap hardware VP8 when a platform backend -advertises it, but this Windows Chrome run selected libvpx and Chromium 151's -Windows Media Foundation backend advertises VP9, H.264, AV1, and conditional -HEVC rather than VP8. A website can prefer a codec/profile but cannot require a -particular hardware implementation. Hardware VP9 or AV1 may provide usable rate -control on supported machines, while H.264/HEVC remain important hardware paths, -but none currently combines VP8's WebRTC baseline coverage with deterministic -hardware selection across Screener's browser targets. Capability advertisement -alone is not acceptance; an actual sender must prove codec, implementation, -power efficiency, rate control, and game-load behavior. - -These results select VP8 for the ordinary Browser path: no-hint VP8 met the -current controlled quality baseline, has the required WebRTC interoperability, -and does not depend on a page choosing a specific Windows encoder. Exact current -source uses `RTCRtpTransceiver.setCodecPreferences()` before -the first offer and fails the edge if the API, a VP8 capability, or the -preference application is missing; it does not leave another video media codec -as fallback. Browser VP8 -hardware acceleration is a separate later evidence question. It must be proven -on the supported browser/platform matrix from the actual encoder and game-load -measurements before it can change the implementation; custom codecs, another -codec ladder, and a Native helper are not implied by this decision. - -Comparable native products do not resolve this browser boundary. Discord -negotiates VP8/H.264 and selected-platform HEVC/AV1 through its own capture and -hardware pipeline, and documents an AMD rate-control/frame-dropper repair. -Parsec, Steam Remote Play, and Moonlight/Sunshine primarily use controlled -hardware H.264/HEVC/AV1 paths. A read-only inspection of the installed Oopz -0.87.425 package found Agora screen sharing, an H.264 Web-viewer configuration, -and native NVENC/QSV/AMF plus software codec paths; it did not prove the codec or -encoder active in a real sender session. These native designs do not prove that -an ordinary web page can force the same encoder path. Reproducibility hashes and -the exact static boundary are retained in -[Native shared encode](./native-shared-encode-sender.md#2026-08-20-windows-codec-comparison). - -Primary implementation evidence: Chromium's -[content-hint bridge](https://chromium.googlesource.com/chromium/src/+/refs/tags/151.0.7922.174/third_party/blink/renderer/modules/peerconnection/media_stream_video_webrtc_sink.cc#36), -libwebrtc's [sender classification](https://webrtc.googlesource.com/src/+/f20ebb8adbf4fa781830e4384c61f732bd28a217/pc/rtp_sender.cc#1419), -[VP8 rate control](https://webrtc.googlesource.com/src/+/f20ebb8adbf4fa781830e4384c61f732bd28a217/modules/video_coding/codecs/vp8/libvpx_vp8_encoder.cc#1360), -Chromium's [hardware codec mapping](https://chromium.googlesource.com/chromium/src/+/refs/tags/151.0.7922.174/third_party/blink/renderer/platform/peerconnection/rtc_video_encoder_factory.cc#75), -its [Windows encoder backend](https://chromium.googlesource.com/chromium/src/+/refs/tags/151.0.7922.174/media/gpu/windows/media_foundation_video_encode_accelerator_win.cc#606), -the [Desktop SW BRC switches](https://chromium.googlesource.com/chromium/src/+/refs/tags/151.0.7922.174/media/gpu/windows/mf_video_encoder_switches.cc#26), -and Chromium's [AMD workaround](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/gpu/config/gpu_driver_bug_list.json#3303). -Product comparisons: [Discord Go Live](https://discord.com/blog/how-it-all-goes-live-an-overview-of-discords-streaming-technology), -[Discord AMD rate control](https://discord.com/blog/from-blocky-to-brilliant-improving-video-quality-on-discord-go-live-on-amd-gpus), -[Oopz privacy policy](https://help.oopz.cn/agreement/privacy), and -[Agora codec selection](https://doc.shengwang.cn/api-ref/rtc/windows/API/enum_videocodectype), -[Parsec compatibility](https://support.parsec.app/hc/en-us/articles/32381568346644-Hardware-and-Software-Compatibility), -[Steam Remote Play update](https://store.steampowered.com/news/posts/?enddate=1734653759&feed=steam_clientAny), -[Moonlight](https://github.com/moonlight-stream/moonlight-qt), and -[Sunshine](https://github.com/LizardByte/Sunshine/blob/master/docs/configuration.md). - -## Evidence Before Adaptation - -Verified specification facts: the W3C stats model supports the needed -separation but does not produce the product decision itself. -`MediaStreamTrack.getSettings()` supplies the track's current configured -dimensions/frame rate, not the recent frame cadence. `RTCVideoSourceStats` -supplies the last-second FPS actually fed to the encoder. Outbound RTP exposes -emitted dimensions/FPS, byte and retransmission counters, encode time, target -bitrate when available, and the current `qualityLimitationReason`; the selected -candidate pair can expose RTT and available outgoing bitrate. Inbound RTP -exposes received dimensions/FPS, byte/loss/jitter counters, decoded/dropped -frames, and freeze counters when implemented. Stats members may be absent, and -cumulative values must be compared across two samples rather than treated as -interval values. - -The user-facing loss value should therefore be an RTP interval loss rate, not -the cumulative `packetsLost` counter and not a claim about UDP itself. For one -unchanged RTP stats object, compute `lostDelta / (receivedDelta + lostDelta)`; -an initial sample, reset, negative delta, zero denominator, or absent field is -unknown. `RTCReceivedRtpStreamStats` defines both counters and notes that -`packetsLost` can be negative. `RTCIceCandidateStats.address` is privacy-sensitive -and remote addresses are null by default unless the application supplied the -candidate. Screener may show a selected address locally in collapsed details, -but it must never enter Viewer C, signaling, logs, exports, persistence, route -selection, or NAT/geography inference. - -Correlate one time interval and media generation across: - -| Evidence | Fields | -| --- | --- | -| A. Host capture | configured width, height, FPS from `getSettings()` plus observed `media-source` input FPS | -| B. Host outbound | RID, width/FPS/bitrate, target/available bitrate, interval encoded frames/encode time, limitation reason, path, RTT, loss/retransmission, and derived negotiated codec/profile/parameters plus applicable `scalabilityMode` | -| C. Viewer inbound | width/FPS/bitrate, loss, jitter, interval decode/drop/freeze, corresponding derived codec/profile/parameters/layer, and actual decode behavior | - -Product inference from those facts: use the following ordered classification: - -| Correlated observation | Likely boundary | -| --- | --- | -| Capture low | capture or constraints | -| Capture high; outbound low; `cpu` | encoder/resource pressure | -| Capture high; outbound low; `bandwidth` | GCC, uplink, or current-path pressure | -| Outbound healthy; inbound low | transport or receiver path | -| Inbound healthy; image visibly blurry | bitrate/quantization, codec, or display scaling | - -This is a diagnostic classification, not a weighted health score. Missing or -reset evidence remains unknown and rebases the interval. - -The smallest implementation sequence is local A+B correlation in one host -sampling tick, followed by a minimal authenticated C report carrying the -receive/decode and derived negotiation signals needed by the topology -predicate. It never carries raw SDP, raw stats, candidate addresses, or raw -device/network identifiers. Server-authoritative path and connection generations -provide authorization and correlation; a general remote stats stream or -telemetry pipeline is unnecessary. - -The repository implements the local host A+B foundation: same-tick track -settings, linked `media-source` input FPS, and one uniquely matched outbound RTP -sample with its RID, explicit sample/media identity and adjacent frame/encode -time deltas, `remoteId` linkage, and the selected path reached through that RTP -stream's transport. Source replacement blocks sampling and invalidates in-flight -generations. Direct/relay edges bind that evidence to the -current track and PeerConnection. The Host SFU publisher also samples only its -exact active LiveKit sender and binds the result again to the -current publication generation; replacement, profile reset, retirement, -disconnect, and authoritative resync clear the old identity before another -sample can appear. The Host details UI renders that source once as `SFU 发送`, -not once per SFU-fed Viewer. This is local diagnostics only: it creates no wire, -score, selector, or media action. - -Authenticated Viewer C is also implemented for each current ordinary or -peer-assisted P2P hop. A viewer sends one nullable, sanitized aggregate window -on the existing two-second stats cadence, bounded to 2 KiB; the server derives -the current viewer, parent, connection, and route revision, while the parent -accepts only the matching local generation and expires it. The report contains -no raw SDP, stats, candidates, addresses, device identifiers, or room identity, -is neither stored nor used for media or routing action, and fails closed for an -SFU-fed root until a real SFU last-hop B/generation exists. - -Codec evidence follows only that outbound RTP object's `codecId`, and the -referenced `RTCCodecStats` must use the same transport. The local diagnostics -expose nullable, bounded MIME plus a normalized profile token and an ASCII allowlist of -format parameters: H.264 `profile-level-id`, `packetization-mode`, and -`level-asymmetry-allowed`; VP9 `profile-id`, `max-fr`, and `max-fs`; VP8 -`max-fr` and `max-fs`; and AV1 `profile`, `level-idx`, and `tier`. Unknown, -invalid, duplicated, oversized, or non-allowlisted fields such as H.264 -parameter sets are discarded. This diagnostic path does not expose or persist -raw `sdpFmtpLine`, SDP, candidates, or complete stats, and it does not infer -specification defaults. Profile tokens remain self-describing, for example -H.264 `profile-level-id=42e01f`; they are not translated into quality, -hardware, support, or capability conclusions. - -The same unique outbound object supplies nullable current configured -`scalabilityMode`. Sender parameter readback exposes an applied mode only for -one unambiguous encoding. Current quality settings do not request a mode, so -the requested value remains null and a browser-reported default is not called -a mismatch; multiple encodings remain unknown. Inbound stats provide no current -standard `scalabilityMode` source, so C does not carry a null-only placeholder. -The earlier exact-production gate rejected its ordered `q,h` publication because -the active lower representation reduced `HIGH`. Current source implements the -later accepted LiveKit-owned two-representation target; production remains the -exact no-RID single-`HIGH` baseline until cutover. - -Official W3C text checked 2026-08-19 defines names ending in `Id` as stats-object -references. In particular, outbound [`mediaSourceId`](https://www.w3.org/TR/webrtc-stats/#dom-rtcoutboundrtpstreamstats-mediasourceid) -references the sender's current media source, whose -[`trackIdentifier`](https://www.w3.org/TR/webrtc-stats/#dom-rtcmediasourcestats-trackidentifier) -is the `MediaStreamTrack.id`; RTP [`transportId`](https://www.w3.org/TR/webrtc-stats/#dom-rtcrtpstreamstats-transportid) -references its transport, and that transport's -[`selectedCandidatePairId`](https://www.w3.org/TR/webrtc-stats/#dom-rtctransportstats-selectedcandidatepairid) -references its selected pair. Therefore an ambiguous RTP set or a broken or -cross-transport reference remains unknown; an opaque stats ID is not a safe -substitute for those relationships. - -The same specification defines [`codecId`](https://www.w3.org/TR/webrtc-stats/#dom-rtcrtpstreamstats-codecid) -as the RTP stream's reference to `RTCCodecStats`, whose `sdpFmtpLine` contains -format-specific negotiated parameters, and defines outbound -[`scalabilityMode`](https://www.w3.org/TR/webrtc-stats/#dom-rtcoutboundrtpstreamstats-scalabilitymode) -as present only when a mode is currently configured for that stream. -WebRTC-SVC separately permits post-negotiation `getParameters()` to return a -different configured mode from one explicitly requested. If no mode was -provided, its absence must not be replaced with an implementation-dependent -default. Chromium currently maps its configured per-stream mode into the -outbound stats field; other browser implementations may omit any optional -member, which remains null rather than a capability conclusion. - -## LiveKit Native Representation Boundary - -ADR-0007 owns the accepted target: the pinned client's default VP8 screen-share -simulcast pair with representation activation and per-subscriber selection -delegated to LiveKit. The pinned gate passed with explicit server send-side BWE. -Current source implements the target; production remains the not-yet-migrated -single-encoding baseline. - -The candidate assumed one shared dual-representation SFU publication could -preserve full resolution while LiveKit selected a lower representation per -constrained subscriber. Static review established two limits before the runtime gate: -ordinary WebRTC receivers have no portable per-RID selection API across -separate PeerConnections. Runtime evidence confirmed that LiveKit 1.13.5 -Dynacast cumulatively enables qualities below the highest requested quality, so -a `HIGH` subscriber keeps `LOW` active. The earlier exact-production A/B proved -that this has a material cost; the later constrained-subscriber gate proved that -the lower representation is necessary and that LiveKit's send-side BWE selects -it without a Screener controller. Sender/viewer evidence remains diagnostic and -does not change topology or command a representation. - -### Historical SFU Source Gate - -The 2026-08-22 source-only gate verified ordered `q,h` configuration and existing -SFU route/admission accounting, but did not start LiveKit or a Browser. The -2026-08-25 exact-production gate supersedes that candidate's quality verdict; -Git history retains the abandoned shaping plan. - -SVC is the third static no-go, -`no-go-web-svc-cross-path-hardware-contract`. WebRTC-SVC configures an outgoing -`RTCRtpSender`; `RTCRtpReceiver` has no matching `setParameters()` experiment or -standard base/enhancement selector. That lets an SFM/SFU selectively forward -layers, but it does not let separate direct/peer PeerConnections consume one -portable shared encoding and independently select layers. Post-negotiation -`getParameters()` can read the currently configured mode when one was -requested, including a browser-selected replacement, but a successful request -does not reveal whether the encoder is hardware or software. - -The remaining capability signals cannot enforce the product's no-silent- -software-fallback rule. Media Capabilities defines `powerEfficient` as an -optimal-power judgment left to the user agent and explicitly notes that -software can qualify. RTCStats exposes `encoderImplementation` and -`powerEfficientEncoder` only when hardware exposure is allowed; the latter -should reflect acceleration but may use other information. These fields are -valuable diagnostics, not an affirmative portable hardware contract. - -Official implementation surfaces checked 2026-08-19 are deliberately treated -as boundaries, not UA quality rankings: - -| Implementation | Static SVC surface | -| --- | --- | -| Chromium/Chrome | Chrome 111 shipped outgoing-track SVC selection; current Blink WebIDL exposes `scalabilityMode` behind its runtime feature. Exact codec/mode and hardware use remain runtime outcomes. | -| Firefox | Current Firefox WebIDL omits `scalabilityMode`; Mozilla's implementation issue remains assigned and behind a preference. | -| Safari/WebKit | Current WebKit WebIDL omits the standard member. LiveKit 2.22.0 carries a Safari-specific legacy encoding branch, but that is an SDK workaround rather than a portable standard setter/readback contract. | - -Pinned LiveKit can select layers per SFU subscriber: client 2.22.0 exposes -`RemoteTrackPublication.setVideoQuality()`, and server 1.13.5 maps each -subscriber's quality/dimensions/FPS to maximum spatial/temporal layers on that -subscriber's downtrack. The actual selector remains codec-dependent: VP8 has a -temporal selector, whereas H.264/H.265 simulcast is spatial-only. That useful -SFU contract still does not extend to direct/peer receivers. Screener's current -SFU subscriber calls `setSubscribed(true)` and requests -`setVideoQuality(HIGH)`. `HIGH` is a ceiling, not a guarantee that BWE forwards -the high layer, so the application does not own the actual layer choice. - -For this screen-share product the pin has an additional hard mismatch. Client -2.22.0 overwrites SVC screen-share publication to `L1T3`, even when another -mode was supplied: one spatial resolution and three temporal layers. It -therefore supplies no low-resolution spatial base and does not satisfy the -accepted LOW/HIGH contract. LiveKit also documents that Dynacast can pause only an entire SVC -stream, not individual SVC layers. Changing those contracts would require a -different dependency/native design decision, not a runtime proof of the pinned -path. No SVC harness was written and no Chrome run was performed. None of the -rejected standard paths may bypass PR #28's stock-GCC/RTX stop line. - -## Why Offline Encoding Presets Do Not Transfer - -[Maruko Toolbox](https://github.com/wzxjohn/marukotoolbox) is an Apache-2.0 -GUI around offline video-processing tools. Its x264-style CRF, slow presets, -lookahead, and multi-pass workflows trade encoding time and buffering for file -size and quality. Interactive WebRTC cannot buffer or revisit future frames in -that way. The transferable principle is to treat bitrate, image detail, frame -rate, and encode cost as a joint tradeoff; its concrete offline presets are not -a WebRTC configuration baseline. -The GUI repository is Apache-2.0, while x264 itself uses GPL-2.0 or commercial -licensing; no implementation or preset is copied into Screener. - -Discord's published Go Live design reaches the same broad conclusion for -realtime game streaming: bandwidth estimates, frame delivery, image quality, -latency, CPU, and memory conflict and must be balanced. Its native encoder -tuning and hardware integration are not available to a browser-only sender and -must not be presented as settings this project already has. - -## Current Source Policy - -The three user-visible profiles remain ceilings rather than promised rates: - -| Profile | Capture ceiling | Sender bitrate ceiling | -| --- | --- | ---: | -| 1080p60 | 1920x1080 at 60 fps | 8 Mbps | -| 1080p30 | 1920x1080 at 30 fps | 5 Mbps | -| 720p30 | 1280x720 at 30 fps | 3 Mbps | - -Current v12 source and production default to the middle `1080p30` ceiling. -Choosing that -default trades a 60 fps ceiling for a 1080p capture bound. The recommended set -remains exactly the three profiles -above. Both expose `480p` only as an advanced `854x480` resolution whose frame rate and bitrate are -selected independently, not as a fourth profile or preset ID. LiveKit currently -uses the same 1080p30 at 5 Mbps screen-share preset, but neither preset -guarantees the emitted resolution, frame rate, or bitrate. - -- Initial capture supplies `ideal` and `max` bounds for resolution and frame - rate; the actual result is read back from `MediaStreamTrack.getSettings()`. -- A live profile change uses `track.applyConstraints()` and updates every - current sender with `RTCRtpSender.setParameters()`. It does not reopen the - source picker or renegotiate healthy peer connections. -- Current source and production leave the video hint unset so display capture - retains browser screen semantics. Recommended profiles and the advanced - initial value use `balanced`, with explicit `maintain-resolution` and - `maintain-framerate` choices. None promises an emitted resolution or rate. -- `maxBitrate` and `maxFramerate` are ceilings. They are neither minimums nor - target guarantees, and the project does not use SDP bitrate hacks. -- The strict `screener-v12` Share advanced settings panel in current source and - production accepts 480p/720p/1080p/1440p, integer 15-60 fps, 2-12 Mbps, and - the three preferences. Browser media is fixed VP8, with no codec UI, - quality-state field, or wire field. The `480p` choice is only advanced - `854x480`, not a fourth recommended profile. Its 64/128/256 kbps audio - ceiling, default 128, is live-switchable on the existing Opus path. Display - capture does not standardize channel-count or sample-rate control. The peer - receive contract permits Opus `stereo=1;maxaveragebitrate=256000`, paired with pinned - LiveKit's explicit high-quality stereo/forceStereo option; the selected sender - ceiling remains separate from negotiated and observed bitrate. - DTX stays fixed off, RED retains the pinned SDK default, and FEC remains - browser/SDK negotiation, not a control or custom adaptation algorithm. The - full boundary is recorded in `docs/research/browser-screen-audio-quality.md`. -- Every sender update derives from `getParameters()`, calls `setParameters()`, - then reads requested/applied bitrate, frame rate, scale, and preference. - Rejection or browser rewriting is visible rather than console-only. -- A P2P or browser-relay sender is constrained to VP8 before the first offer and - receives its quality profile before that offer. Explicit source or - profile changes use the existing serialized sender-mutation path; accepting - an answer does not trigger an extra whole-profile write. -- One strict room setting is last-wins for current/future peer relays and the - configured SFU publisher. Ordinary P2P keeps that state local and does not add - it to the authenticated wire. -- Pausing sharing disables every track in the current capture stream, producing - black video and silence without closing the room or media connection. - -Current source and production stop at manual bounded controls. -They add no composite score, periodic adjustment, runtime codec switching, SDP -bitrate manipulation, or scene detector. Three consecutive samples of one -non-`none` native `qualityLimitationReason` produce one explanatory warning; a -reason change or recovery resets it and never triggers a media action. - -A Chrome 151 loopback smoke with one host, three viewers, and synthetic 720p30 -video propagated balanced then clarity settings to every participant. Every -baseline active outbound video sender displayed the matching requested/applied -preference, peer-connection fingerprints stayed unchanged, and every viewer's -decoded-frame and foreground `requestVideoFrameCallback` counters grew after -each change. Host media edges peaked at two and relay edges at one. This verifies -controls and continuity, not visual quality, full-resolution performance, -CPU/GPU cost, public networks, or sustained behavior. - -## Deliberate Non-Goals - -- No canvas pixel-difference detector, machine-learned rate controller, or - periodic profile switching. -- No copied x264 CRF/preset recipe in the browser path. -- Browser video is fixed VP8, with no runtime, automatic, or user-selected codec - switching. -- No channel-count, sample-rate, codec, arbitrary bitrate, stereo, DTX, RED or - FEC control, and no inference of actual stereo or sample rate from - `opus/48000/2`. Screen media uses one route-consistent stereo contract; - microphone voice, if ever accepted, remains a separate track and policy. -- No promise that all browsers honor `applyConstraints` or degradation - preference identically. -- No automatic composite quality score or periodic profile controller before - the controlled production capture identifies a reproducible bottleneck. -- No room-wide minimum-of-viewers target, per-viewer encoder, UA/device quality - ranking, or silent software SVC fallback. - -## Verification Gate - -Use the same static UI scene and deterministic high-motion game scene at -720p30, 1080p30, and 1080p60. Correlate A capture, B outbound, and C inbound at -the same interval. Record actual dimensions/FPS/bitrate, limitation reason, -codec, encoder implementation, interval encode/decode cost, drops/freezes, -jitter, RTT, loss/retransmission, selected direct/SFU path, and host CPU/GPU. -A live profile change must preserve peer connection IDs, avoid a second source -prompt, and visibly converge to the requested bounds. - -Compute per-frame encode and decode cost from adjacent samples of cumulative -`totalEncodeTime`/`totalDecodeTime` and frame counters. The first sample, a -zero-frame interval, changed stats object, or a counter reset is unknown and -establishes a new baseline. Skip overlapping sampling ticks on the same peer so -reports cannot complete out of order. Do not use a connection-lifetime average -to judge a later overload. Use browser WebRTC diagnostics plus a small run -manifest for time series and percentiles rather than adding a server telemetry -pipeline. - -Compare image readability and motion continuity instead of declaring success -from FPS alone. If reproducible evidence later shows a supported preference -oscillates or makes the wrong tradeoff on supported machines, revise that -explicit option only from a new controlled gate. The exact Browser SFU gate -showed one VP8 outbound video encoding, no `q,h` simulcast, and continuing -publisher/subscriber frame progress without changing ADR-0005 resource accounting. -The application's evidence windows remain diagnostic only; they do not command -another representation, change topology, or override stock congestion control. +Current parent selection hard-filters authorization, source reachability, +acyclicity, capacity, and resource admission, then orders eligible P2P parents +by resulting depth, remaining sender capacity, stable join order, and peer +identity before SFU. It has no network-quality score. + +A manual media reconnect stays on the current exact route: P2P rebuilds the +same parent connection, and SFU reconstructs the same subscription. Browser +page refresh also rebinds the same stable participant and committed graph. +Only actual recovery exhaustion enters the normal route-failure operation and +tries other eligible P2P parents before SFU. + +Quality-driven relay abdication or active parent switching would require +counterfactual alternative-path evidence, bounded probing, and a separate +ADR-0005 decision. It is not implemented or authorized by current diagnostics. + +## Current Verification Gaps + +- Real games under competing CPU/GPU load at 720p30, 1080p30, and 1080p60. +- Weaker Windows/macOS/Linux Hosts and actual VP8 encoder implementation data. +- Mixed P2P/SFU sessions on heterogeneous public networks, including recovery. +- Mobile foreground/background playback, page reclamation, and relay survival. +- Long-running encode/decode cost, thermals, A/V synchronization, and resource + admission at the 20-Viewer bound. + +Measurements must correlate capture, outbound, and inbound stats over the same +interval. Missing counters and identity changes remain unknown, not zero. ## Primary Sources -- [W3C MediaStreamTrack Content Hints](https://www.w3.org/TR/mst-content-hint/) -- [W3C Screen Capture](https://www.w3.org/TR/screen-capture/) -- [W3C Media Capture and Streams `getSettings()`](https://www.w3.org/TR/mediacapture-streams/#dom-mediastreamtrack-getsettings) -- [W3C WebRTC](https://www.w3.org/TR/webrtc/) -- [W3C WebRTC codec preferences](https://www.w3.org/TR/webrtc/#dom-rtcrtptransceiver-setcodecpreferences) -- [W3C WebRTC sender track replacement](https://www.w3.org/TR/webrtc/#dom-rtcrtpsender-replacetrack) -- [Jitsi codec preference and renegotiation](https://jitsi.github.io/lib-jitsi-meet/classes/JitsiConference._internal_.JingleSessionPC.html) -- [RFC 7742 WebRTC video codec requirements](https://www.rfc-editor.org/rfc/rfc7742.html) -- [Via LA AVC/H.264 licensing program](https://via-la.com/licensing-programs/avc-h-264/) -- [W3C WebRTC Statistics](https://www.w3.org/TR/webrtc-stats/) -- [W3C WebRTC SVC](https://www.w3.org/TR/webrtc-svc/) -- [W3C Media Capabilities](https://www.w3.org/TR/media-capabilities/) -- [W3C WebCodecs](https://www.w3.org/TR/webcodecs/) -- [Chrome 111 WebRTC SVC extension](https://developer.chrome.com/blog/chrome-111-beta) -- [Chromium `RTCRtpEncodingParameters` WebIDL](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/third_party/blink/renderer/modules/peerconnection/rtc_rtp_encoding_parameters.idl) -- [Firefox `RTCRtpEncodingParameters` WebIDL](https://searchfox.org/firefox-main/source/dom/webidl/RTCRtpParameters.webidl) -- [Firefox WebRTC-SVC implementation status](https://bugzilla.mozilla.org/show_bug.cgi?id=1571470) -- [WebKit `RTCRtpEncodingParameters` WebIDL](https://github.com/WebKit/WebKit/blob/main/Source/WebCore/Modules/mediastream/RTCRtpEncodingParameters.idl) -- [RFC 6184 H.264 RTP payload format](https://www.rfc-editor.org/rfc/rfc6184.html) -- [RFC 7741 VP8 RTP payload format](https://www.rfc-editor.org/rfc/rfc7741.html) -- [RFC 9628 VP9 RTP payload format](https://www.rfc-editor.org/rfc/rfc9628.html) -- [IANA AV1 media type and format parameters](https://www.iana.org/assignments/media-types/video/AV1) -- [Chromium/libwebrtc video stats origins](https://webrtc.googlesource.com/src/+/HEAD/video/g3doc/stats.md) -- [Chromium WebRTC hardware encoder factory](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/third_party/blink/renderer/platform/peerconnection/rtc_video_encoder_factory.cc) -- [Chromium Windows H.264 constrained-baseline acceleration gate](https://github.com/chromium/chromium/blob/main/third_party/blink/renderer/platform/peerconnection/webrtc_util.cc) -- [Chromium Windows Media Foundation encoder selection](https://chromium.googlesource.com/chromium/src/+/master/media/gpu/windows/media_foundation_video_encode_accelerator_win.cc) -- [Chromium WebRTC hardware fallback and simulcast initialization](https://chromium.googlesource.com/chromium/src/+/HEAD/third_party/blink/renderer/platform/peerconnection/rtc_video_encoder.cc) -- [Chromium WebRTC AV1 hardware feature gate](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/media/webrtc/webrtc_features.cc) -- [MDN `RTCRtpSender.setParameters()`](https://developer.mozilla.org/en-US/docs/Web/API/RTCRtpSender/setParameters) -- [Chromium `motion` to libwebrtc `kFluid` bridge](https://chromium.googlesource.com/chromium/src/third_party/+/refs/heads/main/blink/renderer/modules/peerconnection/media_stream_video_webrtc_sink.cc) -- [libwebrtc `motion`/`kFluid` sender classification](https://webrtc.googlesource.com/src/+/3b1eab8a69cb5078befb021c5492d3f204a7d6a2/pc/rtp_sender.cc) -- [libwebrtc pre-negotiation sender parameter storage](https://webrtc.googlesource.com/src/+/refs/heads/main/pc/rtp_sender.cc) -- [libwebrtc encoder content-type selection](https://webrtc.googlesource.com/src/+/9caef2a8b88f389af10cee841732c42a98d3d45d/media/engine/webrtc_video_engine.cc) -- [libwebrtc conditional screen-share degradation mapping](https://webrtc.googlesource.com/src/+/refs/heads/main/video/video_stream_encoder.cc) +- [MediaStreamTrack Content Hints](https://www.w3.org/TR/mst-content-hint/) +- [Screen Capture](https://www.w3.org/TR/screen-capture/) +- [WebRTC](https://www.w3.org/TR/webrtc/) +- [WebRTC Statistics](https://www.w3.org/TR/webrtc-stats/) - [libwebrtc adaptation overview](https://webrtc.googlesource.com/src/+/HEAD/video/g3doc/adaptation.md) -- [libwebrtc per-send-stream encoder construction](https://webrtc.googlesource.com/src/+/refs/heads/main/video/video_send_stream_impl.cc) -- [libwebrtc removal of the automatic animation-detection experiment, 2024-05-22](https://webrtc.googlesource.com/src/+/1d7d0e6e2c5002815853be251ce43fe88779ac85) -- [LiveKit screen-share presets](https://github.com/livekit/client-sdk-js/blob/main/src/room/track/options.ts) -- [LiveKit degradation defaults](https://github.com/livekit/client-sdk-js/blob/main/src/room/participant/publishUtils.ts) +- [LiveKit screen-share encoding construction](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/participant/publishUtils.ts) - [LiveKit video simulcast and Dynacast](https://docs.livekit.io/transport/media/advanced/) - [LiveKit selective subscription](https://docs.livekit.io/transport/media/subscribe/) -- [LiveKit client 2.22.0 room defaults](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/defaults.ts) -- [LiveKit client 2.22.0 room and Dynacast options](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/options.ts) -- [LiveKit client 2.22.0 room option merge](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/Room.ts) -- [LiveKit client 2.22.0 SVC defaults](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/track/options.ts) -- [LiveKit client 2.22.0 screen-share SVC, Dynacast enablement, and republish lifecycle](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/participant/LocalParticipant.ts) -- [LiveKit client 2.22.0 start-bitrate negotiation](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/PCTransport.ts) -- [LiveKit client 2.22.0 SVC encoding construction](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/participant/publishUtils.ts) -- [LiveKit client 2.22.0 subscriber quality control](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/track/RemoteTrackPublication.ts) -- [LiveKit server 1.13.5 per-subscriber layer application](https://github.com/livekit/livekit/blob/v1.13.5/pkg/rtc/subscribedtrack.go) -- [LiveKit server 1.13.5 codec-specific layer selectors](https://github.com/livekit/livekit/blob/v1.13.5/pkg/sfu/forwarder.go) -- [LiveKit client 2.22.0 Dynacast and saved degradation preference](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/track/LocalVideoTrack.ts) -- [LiveKit client 2.22.0 local sender stats](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/track/LocalTrack.ts) -- [LiveKit server 1.13.5 Dynacast quality aggregation](https://github.com/livekit/livekit/blob/v1.13.5/pkg/rtc/dynacast/dynacastqualityvideo.go) -- [LiveKit server 1.13.5 enabled-quality generation](https://github.com/livekit/livekit/blob/v1.13.5/pkg/rtc/dynacast/dynacastmanagervideo.go) -- [LiveKit server 1.13.5 release assets and checksums](https://github.com/livekit/livekit/releases/tag/v1.13.5) -- [Microsoft `New-NetQosPolicy`](https://learn.microsoft.com/en-us/powershell/module/netqos/new-netqospolicy) -- [Linux network namespaces](https://man7.org/linux/man-pages/man7/network_namespaces.7.html) -- [Linux `tc-tbf`](https://man7.org/linux/man-pages/man8/tc-tbf.8.html), [`tc-netem`](https://man7.org/linux/man-pages/man8/tc-netem.8.html), and [`tc-flower`](https://man7.org/linux/man-pages/man8/tc-flower.8.html) -- [Jitsi desktop degradation preference](https://github.com/jitsi/lib-jitsi-meet/blob/master/modules/RTC/TraceablePeerConnection.ts) -- [Discord Go Live architecture](https://discord.com/blog/how-it-all-goes-live-an-overview-of-discords-streaming-technology) -- [Discord encoder-quality case study](https://discord.com/blog/from-blocky-to-brilliant-improving-video-quality-on-discord-go-live-on-amd-gpus) -- [NVIDIA NVENC application note](https://docs.nvidia.com/video-technologies/video-codec-sdk/13.1/nvenc-application-note/index.html) -- [Agora Windows screen sharing](https://doc.shengwang.cn/doc/rtc/windows/basic-features/screen-share) -- [Agora Windows encoding preference](https://doc.shengwang.cn/api-ref/rtc/windows/API/enum_encodingpreference) -- [Oopz help center](https://help.oopz.cn/) +- [LiveKit server forwarding](https://github.com/livekit/livekit/blob/v1.13.5/pkg/sfu/forwarder.go) +- [Chromium WebRTC encoder factory](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/third_party/blink/renderer/platform/peerconnection/rtc_video_encoder_factory.cc) diff --git a/docs/research/webrtc-p2p-screen-sharing.md b/docs/research/webrtc-p2p-screen-sharing.md index d9b4761..ef2d746 100644 --- a/docs/research/webrtc-p2p-screen-sharing.md +++ b/docs/research/webrtc-p2p-screen-sharing.md @@ -160,7 +160,7 @@ parameters.degradationPreference = "balanced"; await sender.setParameters(parameters); ``` -这些参数是偏好或上限，不能绕过浏览器拥塞控制，也不能保证目标码率。`balanced` 允许浏览器在分辨率和帧率之间权衡，但不规定具体算法；视频 track 不设置 `contentHint`，让 Chromium/libwebrtc 保留 display capture 的 screen 分类。当前源码链与测量要求见 [Realtime quality adaptation](./realtime-quality-adaptation.md)。静态画面通常能降低编码数据量，但规范不保证所有浏览器主动降低捕获频率或 GPU 开销。 +这些参数是偏好或上限，不能绕过浏览器拥塞控制，也不能保证目标码率。`balanced` 允许浏览器在分辨率和帧率之间权衡，但不规定具体算法；视频 track 使用标准 `contentHint = "motion"` 表达游戏动态意图。当前源码链与测量要求见 [Realtime quality adaptation](./realtime-quality-adaptation.md)。静态画面通常能降低编码数据量，但规范不保证所有浏览器主动降低捕获频率或 GPU 开销。 因此首版不实现画面差分检测、周期性 `applyConstraints()` 或自定义动态 FPS 状态机。先对静态桌面和高动态游戏分别记录 `framesEncoded`、发送码率、`totalEncodeTime / framesEncoded`、`qualityLimitationReason` 及主机 CPU/GPU 占用；只有测量显示浏览器行为留下显著问题时，再设计最小的控制策略。这避免用额外竞态、计时器和画质跳变解决一个可能已由捕获器、编码器和 WebRTC 拥塞控制处理的问题。 diff --git a/docs/session-handoff.md b/docs/session-handoff.md index d7a48a3..25ca553 100644 --- a/docs/session-handoff.md +++ b/docs/session-handoff.md @@ -2,82 +2,28 @@ Last updated: 2026-08-25 -This file is a bootstrap prompt, not a product-truth owner. The linked documents -replace any duplicated or conflicting statement here. Update or remove this file -after the handoff boundary changes. +This file is only a bootstrap pointer. Product truth lives in the linked owners. -## Prompt +## Resume Prompt ```text -Continue the Screener project from the canonical repository root: +Continue Screener from the canonical repository root: C:\Users\TNTcraft\Documents\GitHub\Screener -Before changing code, deployment, branches, worktrees, or TODO state: +Read AGENTS.md and the complete repo-tracked stop-that-shit skill first. Then +read CONTRIBUTING.md, docs/project-memory.md, docs/status.md, docs/todo.md, the +relevant requirement/design/ADR/research owner, and inspect branch/HEAD, status, +staged diff, and worktrees. Canonical main and repo truth override old branches, +worktrees, handoffs, agent memory, and chat summaries. -1. Read the root AGENTS.md instructions supplied for this repository, then read - CONTRIBUTING.md, docs/project-memory.md, docs/status.md, docs/todo.md, - docs/需求理解.md, docs/方案设计.md, docs/maintenance.md, and the relevant - ADR/research owner. -2. Inspect git status, current branch/HEAD, worktree list, and the diff. Preserve - every pre-existing edit. The canonical root/main owns truth; auxiliary - worktrees, branches, agent memories, chat summaries, and old TODOs do not. -3. Distinguish accepted product truth, current source, and current production. - Do not report an accepted design as implemented or deployed. -4. Resolve meaning and update all affected truth owners before implementation. - Keep the model simple, use existing framework behavior, avoid case-by-case - patches, and do not add compatibility for unpublished internal releases. +Current production and main identity is owned by docs/status.md. ADR-0002 owns +memory rooms; ADR-0005 owns automatic direct/peer-to-LiveKit routing; ADR-0007 +owns framework media adaptation. Current routes are sticky and quality evidence +is diagnostic only. Browser video is VP8 with contentHint=motion; LiveKit owns +SFU representations; Screener configures no TURN or media TCP. -Current high-signal boundary: - -- docs/status.md owns the exact source/production/hold state. -- docs/todo.md is the only executable work ledger and owns ordering. -- ADR-0005 owns the accepted automatic route model: direct/STUN peer followed - only by the dedicated LiveKit SFU/UDP fallback. Browser SFU publisher and - subscriber PCs explicitly use no external ICE server and check the dedicated - public SFU candidate directly; ordinary peers and LiveKit server-side public - IP discovery retain deployment STUN. Production exact deployed - application/runtime revision `1d8761528d0dba43fb6d818df3934483ba2f5340`, - release `1d87615`, uses strict `screener-v12` and includes exact Browser SFU - ICE-server-isolation implementation - `ae09c760adec76fd26da611d4928486d105c6d3b`. Chrome 151 on that exact production - verified one no-RID publisher encoding and progressing publisher/subscriber - frames over LiveKit UDP. The route uses one short-lived - exact-generation decoded-frame observer only for the pending candidate inside - the existing total deadline; the periodic stats sampler still owns active-path - diagnostics and stalls. Broader external-network and SFU lifecycle evidence - remains open. Do not run executable/native suites unless their later acceptance - boundary specifically requires them. -- ADR-0002 owns the room model. Current source and production use strict v12 with - exactly `open | private` code entry and the room-lived 22-character Viewer - grant. Neither source nor production uses SQLite; exact release operations and - change-scoped infrastructure recovery are owned by deployment. -- Browser display video must leave `contentHint` unset; audio keeps `music`. - Browser video is fixed VP8 across direct, browser-relay, and SFU paths; strict - v12 source and production have no codec UI/state/wire or fallback media codec. - Automated source gates and the exact deployment postflight passed. Chrome 151 - physically verified direct/browser-relay/SFU screen-audio continuity and all - three sender ceilings. An exact-production VP8 gate sustained about 59.5 fps - on direct and Browser-relay paths and rejected the then-deployed `bf32859` - SFU `q,h` publication because active `LOW` reduced `HIGH`. ADR-0007 accepts - LiveKit-owned half-resolution plus original-resolution simulcast, Dynacast, - and server send-side BWE. Current source implements that combination while - production remains single-encoding; Chrome - 151 verified the production no-RID baseline and separately proved the accepted - native adaptation on the pinned stack. -- Ordinary browser ICE owns direct reachability. Do not build port prediction, - guessed candidates, NAT classification, TCP probes, quality scores, or - quality-driven reparenting. Current-path quality is diagnostic; a healthy - decoded edge remains sticky. -- docs/maintenance.md owns the reusable mechanism-simplification review; - ADR-0002 owns the accepted memory-room tradeoff. The later repository audit in - TODO is not permission for broad cleanup during this handoff. -- The canonical root/main checkpoint reconciles the route truth audit with this - room decision. Use that coherent truth as the base; do not replace it with - older documents from another worktree. -- Existing auxiliary worktrees and branches are retained candidates or audit - material. Do not merge or delete them until their unique changes, references, - and reparse/junction safety are checked against current main. - -Follow the current ledger in order and report each verified physical-evidence -boundary without reviving stale agent or branch facts. +Follow docs/todo.md in order. Do not run Native/executable work, broad repository +cleanup, quality-driven reparenting, or another transport unless its decision +boundary is explicitly opened. Preserve unique or dirty worktrees until merged- +head, reference, and reparse audits prove cleanup safe. ``` diff --git a/docs/status.md b/docs/status.md index ebf342f..e9fb7dc 100644 --- a/docs/status.md +++ b/docs/status.md @@ -2,47 +2,78 @@ Last updated: 2026-08-25 -This is the current execution index. Git history owns completed timelines; [verification status](./verification-status.md) owns evidence boundaries. +This is the current execution index. Git history owns completed timelines; +[verification status](./verification-status.md) owns evidence boundaries. ## Production -- `https://share.bonfire.icu` runs exact deployed application/runtime revision `be53c4d5794d38d5d406c876309958d7117ee601`, release `be53c4d`, wire `screener-v12`, from `/opt/screener/releases/be53c4d`; that revision is integrated in canonical `main`. Its immutable runtime tar SHA-256 is `5c174c30ffacd98738a364b825b915b84432823c261272684937361be2451144`, and its 39-file manifest SHA-256 is `c34dba43a25c6b8447e1153a5be3191bf82cff94117ed31edc325d3f7cf8541f`. -- `screener`, LiveKit, coturn, and nginx are active/running; local/public `/healthz` and the served Browser asset match the immutable release, and all four services report `NRestarts=0`. The main Browser asset is `assets/index-BKEVI7Ux.js` with SHA-256 `22114fc8a26b3f03f37973555433d1e73d52d1c7b76fa3c7cc597500823bd62d`; predecessor asset `assets/index-Bjcb0_oa.js` returns 404. -- The service process resolves its working directory to exact release `be53c4d`; it has no writable room `StateDirectory` or SQLite runtime. -- Production admits one Host plus 20 Viewers, applies endpoint capacity `2`, rejects stale Browser and executable-sender wires before room authority, and runs the one-controller exact-candidate route model with one staged total deadline. Rooms use random free four-digit codes, a 24-hour dormant lease, restart loss, local Host creation preferences, a room-lived 22-character Viewer grant, and exactly `open | private` code entry. Private rooms without a password are invitation-only; configuring one additionally permits matching code-and-password entry. LiveKit remains dedicated with `room.auto_create: false`, `max_participants: 21`, and global admission `1` publication ingress / `20` subscription egress. -- Production media ports remain STUN-only UDP 3478 and LiveKit UDP 7882; coturn TCP/TLS, port 5349, media TCP, and relay ranges are disabled. The public Web ingress remains TCP 80/443; Node 8787 and LiveKit control/signaling 7880 are internal-only. The stable `inet bonfire_filter` table SHA-256 is `afaf9d066e9f292d8bd19032b38c0978ccf60dd01c2fe7f210968ff751f90534`; the dynamic fail2ban table is excluded from that identity. -- SFU subscribers reconcile Host screen publications across connect, activation, participant arrival, track publication, and reconnect. LiveKit runs at `warn`/Pion `error`, and the Screener unit waits boundedly for the same-host LiveKit control listener before startup. The decoded-progress cadence correction is deployed; the earlier subscriber-PC closure remains only the reproduction, so a controlled recovery transaction and physical network-loss evidence remain open. -- Production Browser SFU publisher and subscriber PCs explicitly use empty external ICE-server lists while retaining LiveKit-signaled UDP candidates. The owner verified that the deployed isolation fixes the reported SFU connection failure; the prior P1 core-route hold is closed. -- Production leaves video `contentHint` unset, keeps audio `contentHint = "music"`, and fixes Browser direct, browser-relay, and SFU video to VP8 without codec UI, quality state, wire fields, or fallback media codecs. Its SFU publisher supplies LiveKit one half-actual-resolution lower preset capped at 30 fps with proportional bitrate beside the original representation, enables Dynacast and server send-side BWE, keeps AdaptiveStream disabled, and adds no Screener layer selector. +- `https://share.bonfire.icu` runs exact application/runtime revision + `af348ee1d508a3af02b18a7f46c461953798e19d`, release `af348ee`, wire + `screener-v12`, from `/opt/screener/releases/af348ee`. The immutable runtime + tar SHA-256 is + `4922cc31d7caaad7c5412b1f2fe1d73f788d16d92386efad6eac358a97b8a78e`; + its 39-file manifest SHA-256 is + `5b397809b664b9a55885ac603df615b537bf51276e7274416820870956242d7d`. +- The served Browser entry references `assets/index-BvuZY6Hc.js`; the public + asset is 488473 bytes with SHA-256 + `e8fc0500496ee101031a9cfebfcc22d3b69e66d2cac5e875448b56620dbb21c5`. + Public `/healthz` returns 200. The release postflight found Screener, LiveKit, + coturn, and nginx active with zero restarts. +- Production uses process-memory four-digit rooms, a 24-hour dormant lease, + room-lived 22-character grants, `open | private` code entry, one Host plus 20 + Viewers, endpoint copy cap `2`, and no SQLite runtime. +- Public media listeners remain STUN-only UDP 3478 and LiveKit UDP 7882. Web + ingress is TCP 80/443; Node 8787 and LiveKit control/signaling 7880 are + private. TURN, ICE/TCP, media TCP, TLS relay, port 5349, and relay ranges are + disabled. +- Browser video is fixed VP8 with `contentHint = "motion"`; audio uses `music`. + The SFU publisher leaves representation construction to pinned LiveKit, + enables Dynacast, uses server send-side BWE, keeps AdaptiveStream disabled, + and configures no external ICE servers on Browser SFU PCs. +- Screen audio provides live 64/128/256 kbps ceilings with 128 default. SFU + publication uses stereo, DTX off, and RED off. Early autoplay presentation is + gated by current media connection state. ## Current Source -- Canonical root `main` is the integration truth; auxiliary branches and older worktrees do not supersede it. Current source and production use the single strict Browser `screener-v12` wire and reject every other wire before room authority. -- Current source implements one Host plus up to `20` Viewers, one steady outbound media-copy cap for every non-server endpoint (default `2`, static `1/2/3`), and the process-memory room model with random four-digit codes, a default 24-hour dormant lease, exact-Host resume, restart loss, local Host preference replay, exactly `open | private` code entry, and no SQLite runtime. Every room defaults to `open` without a room password and independently receives a Viewer grant: one 22-character opaque value bound to that room incarnation. Private rooms without a password are invitation-only; configuring a password additionally permits matching code-only entry. Unallocated/reclaimed/expired code-only entry returns `ROOM_NOT_FOUND` without a password prompt. -- One event-driven controller owns the committed graph and at most one room-serial child operation. It uses one deterministic candidate list/cursor and one total direct-then-SFU deadline; exact admission and physical resources remain charged through drain, and the pending candidate commits only after an exact-generation decoded-frame proof. A short-lived 100 ms stats observer proves only that candidate inside the existing deadline. The active-path 2-second sampler now keeps its existing 15-second decoded-progress cadence when an SFU subscriber has no video track, no merged report, or a failed stats read, without adding a timer or fabricating metrics. Healthy edges remain sticky and current-path quality does not authorize reparenting. -- Ordinary peer ICE is STUN-only and the only application fallback is the dedicated LiveKit SFU over UDP. SFU generations use explicit no-default ingress/egress capacities, exact `reserved | committed | draining` accounting, delete-plus-absence release, one Host publication, and exact Viewer subscription handles. No Screener TURN, ICE/TCP, media TCP, or TLS-relayed media route exists. -- Current Browser SFU publisher and subscriber connects supply fresh `autoSubscribe: false` plus `rtcConfig: { iceServers: [] }` options. Ordinary peers retain deployment STUN; LiveKit signaling candidates, server-side public-IP discovery, route deadlines, ports, and retry behavior are unchanged. -- Current source leaves video `contentHint` unset and keeps audio `contentHint = "music"`. Browser direct and browser-relay offers contain VP8 as their only media codec. SFU publication fixes VP8 with no backup codec and supplies LiveKit one half-actual-resolution lower preset capped at 30 fps with proportional bitrate beside the original representation; Dynacast, the default maximum subscriber ceiling, server send-side BWE, and disabled AdaptiveStream remain unchanged. Codec UI, quality state, wire fields, fallback media codecs, and share-lifetime codec branches are absent. Live 64/128/256 kbps audio-ceiling mutation remains unchanged; ordinary Host and SFU publisher metrics select the exact current sender, relays reconcile audio-track additions and removals on the persistent stream, and an SFU reconnect reacquires the exact current publication/sender, avoids unchanged parameter/stat resets, or fails the route closed. -- Current source implements typed first-frame Viewer presentation and recovery, Chinese user-facing route/access failures, local connection details without a diagnostic-file export, authenticated Host-only on-demand route snapshots for automatic acceptance, explicit missing-room presentation, and responsive entry controls. Host invitation controls remain independent from two-state room-code entry. -- Recommended quality remains exactly `1080p60`, `1080p30`, and `720p30`; current source defaults to `1080p30`. Advanced resolution adds `480p` as `854x480` without adding a fourth preset, and advanced FPS and bitrate remain independent. -- The deployed application tree passed repository hygiene, TypeScript, all 613 Web tests in 44 files, both production builds, two independent reviews, and immutable artifact/postflight checks. Earlier exact-production gates exercised direct, browser-relay, SFU audio continuity and all three audio ceilings, and sustained about 59.5 fps on direct and browser-relay paths; the pinned-stack gate proved native two-representation selection. The new lower representation's mixed-route quality effect still needs production media evidence. Exact applicability remains in [verification status](./verification-status.md). +- Canonical root `main` is clean at the same exact `af348ee` revision as + production. Auxiliary branches and worktrees do not supersede it. +- One event-driven controller owns the committed graph and one room-serial child + operation. It uses exact candidate identity, first-decoded-frame commit, + strict rollback revisions, direct-then-SFU total deadline, and typed endpoint + plus SFU resource accounting. Healthy decoded edges remain sticky. +- WebRTC/LiveKit own media adaptation. Screener has no custom SFU layer list, + quality score, layer selector, periodic rebalancing, or quality-driven parent + change. P2P and SFU recover their current route before actual failure enters + normal reassignment. +- The staged `fix/viewer-page-lifecycle` candidate is based on exact `af348ee`. + It rebaselines Viewer decoded-stall timing after page resume, makes a new + current-generation frame clear stale media recovery state, uses native Viewer + video controls, removes Host preview controls, and makes manual P2P reconnect + rebuild the same parent. It is accepted but not yet committed, merged, or + deployed. ## Current Milestone -1. Verify ADR-0007's deployed lightweight half-resolution representation in a mixed P2P/SFU room, then repeat the controlled active-SFU recovery transaction on exact production and finish representative ICE/STUN/SFU acceptance including screen-audio continuity and real-game A/V sync on mobile and heterogeneous networks. Browser port prediction, NAT classification, TCP probing, fake page keepalive, and quality-driven reparenting remain outside the accepted model. +1. Integrate, validate, deploy, and postflight `fix/viewer-page-lifecycle`. +2. Finish representative public-network direct, peer-relay, SFU, recovery, + Pause/Resume, screen-audio, real-game A/V, mobile lifecycle, and all-UDP- + blocked acceptance without adding another transport or quality controller. ## Active Boundaries -- `fix/configurable-relay-cap` is an old, incomplete draft and must not be merged as-is. -- The desktop Host background/minimized report is deferred until a current-production real-game reproduction supplies synchronized media and CPU/GPU evidence. Mobile Viewer background playback and relay survival remain a separate Accepted Later lifecycle gate. -- Open PR #192 and the Native stack are evidence/research, not pending product releases. Native senders, capture helpers, shared-encode executables, and their test binaries are outside the current Browser milestone and are not built or run. -- Deployed surfaces and retained candidates that still need product decisions are indexed only in [the TODO ledger](./todo.md); do not extend or roll them back automatically. -- Room lifetime, authorization/storage semantics, restart loss, and routing are aligned on strict v12 in current source and production. The application-only cutover reused unchanged infrastructure and configuration and does not maintain a full rollback/configuration backup; any future infrastructure or irreversible-state change requires recovery scoped to the surfaces it actually changes. -- Real SFU recovery, heterogeneous networks, mobile lifecycle, audio/A-V device behavior, and endurance/resource measurements remain external acceptance evidence, not blockers for unrelated reversible work. +- Desktop Host background capture remains unconfirmed. The current-browser + screening did not reproduce a Host-page drop; accepted Viewer lifecycle work + prevents frozen JavaScript wall time from becoming an immediate route failure + but is not capture keepalive. +- Current-path quality is diagnostic. High loss, RTT, jitter, low bitrate, + resolution, FPS, or freeze counters do not trigger relay abdication or parent + switching. +- Native/executable senders, shared encode, distribution packages, broad UI + polish, bilingual support, and repository-wide simplification are outside the + current release. ## Current Hold -The active-SFU cadence correction is deployed, but the controlled recovery -transaction has not yet been repeated on this release. That exact proof remains -the P1 hold for route acceptance. Native/executable work and broad repository -cleanup remain outside the current evidence boundary. +No source or deployment P0 is open. Representative real-network and mobile +physical evidence remains required before route acceptance is complete. diff --git a/docs/todo.md b/docs/todo.md index 014c0ff..a246009 100644 --- a/docs/todo.md +++ b/docs/todo.md @@ -2,45 +2,87 @@ Last reviewed: 2026-08-25 -Only items in **Now** are executable after their stated decision gate. A branch name, unchecked requirement, experiment, review suggestion, or deployed behavior is not a TODO by itself. +Only **Now** is executable. A branch, old experiment, observation, or accepted +later topic is not implementation authority by itself. ## Now -1. **Verify deployed constrained SFU publication quality.** Production keeps the two LiveKit-owned VP8 spatial representations, Dynacast, send-side BWE, and no AdaptiveStream. It derives the half-actual-resolution lower publication preset at no more than 30 fps with proportional bitrate and retains that option for LiveKit-owned source replacement or republish; it does not mutate LOW behind the SDK's Dynacast lock. In the same mixed P2P/SFU dynamic-source gate, verify that HIGH recovers materially, P2P does not regress, and a constrained SFU downlink still selects and decodes LOW. Do not restore global `motion`, add more simulcast encodings, return to single HIGH, or add a Screener layer selector. -2. **Finish real-network route acceptance.** Repeat the controlled active-SFU recovery transaction on exact production. Then measure standard ICE/STUN direct paths, peer relay, SFU ingress/subscription, relay-ingress repair with subtree retention, disconnect/capacity drain, Pause/Resume, screen-audio continuity and real-game A/V sync across representative IPv4/IPv6 NATs and mobile networks. Include Host and Viewer TUN/VPN cases and an all-UDP-blocked case to distinguish an application ICE-option defect from operating-system interception and establish bounded explicit failure. Do not add another watchdog, port prediction, NAT classification, TCP probes, guessed candidates, route scores, or quality-driven reparenting. +1. **Close the Viewer lifecycle/native-control release.** Integrate the staged + `fix/viewer-page-lifecycle` candidate from the accepted truth checkpoint, + run the full repository check, review the exact diff, merge through PR, + deploy one immutable application artifact, and perform scoped production + postflight. Verify native Viewer controls plus current-route P2P and SFU + reconnect without adding route reselection. +2. **Finish real-network route and media acceptance.** Exercise direct peer, + Browser relay, SFU, relay-ingress recovery with subtree retention, + disconnect/capacity drain, Pause/Resume, source replacement, screen-audio + continuity, and real-game A/V sync across representative IPv4/IPv6, + Wi-Fi/cellular, and Host/Viewer TUN/VPN cases. Include an all-UDP-blocked case + and weak-network audio after SFU RED was disabled. Every exhausted path must + end in bounded wait/failure without TURN, TCP probing, NAT classification, + guessed candidates, or another watchdog. ## Accepted Later Roadmap -These items are real product work, ordered after the current physical-media, real-network, and product-decision gates; they are not permission to resume an old branch wholesale. +1. **Browser VP8 hardware evidence.** Measure supported Chrome/Edge platforms + under real game load using actual codec, encoder implementation, encode time, + frame rate, and CPU/GPU attribution. Do not add another codec, custom encoder, + GPU selector, or Native helper without a new decision. +2. **Mobile Viewer lifecycle.** Run Android Chrome and iOS Safari matrices for + autoplay gesture, foreground/background audio, foreground video recovery, + lock/page reclamation, rotation, Wi-Fi/cellular migration, and assigned-relay + survival or controller recovery. Web does not promise background video or + relay execution after OS suspension. +3. **Quality-based topology research.** After the real-network baseline, decide + whether measured user-visible quality justifies any active parent-selection + mechanism. Current evidence cannot compare an active route with an + unconnected alternative, so no quality score, all-pairs probing, periodic + rebalancing, relay abdication threshold, or active parent switch is accepted. + Prefer mature algorithms and one general model if this boundary is reopened. +4. **Platform output only when real.** Revisit AirPlay/Cast only when a target + browser and physical receiver prove the live `MediaStream` contract. System + mirroring remains external. +5. **Public-server one-click package.** After functional and real-network work, + package the exact application, STUN/SFU, reverse proxy, secrets, and health + checks for a user-owned public server. Do not call a partial installer ready. +6. **Fully local one-click package.** Package Windows/macOS/Linux Host capture, + application server, and local state without requiring source or Node. Report + public-origin, TLS, gateway, NAT, and firewall limits honestly. +7. **Native Host and shared encode, Windows first.** Productize only after real + capture, hardware-only encode, audio, identity, RTP/RTCP feedback, resource, + packaging, and licensing gates pass. Browser Host/relay keeps standard + per-`RTCPeerConnection` encoding. +8. **Whole-product UI and bilingual decision.** Once media and route behavior + stabilizes, review copy, responsive hierarchy, visual consistency, restrained + motion, bundle/rendering cost, and Chinese/English scope once as a whole. + Ordinary screen-specific edits do not create parallel documentation. +9. **Repository simplification audit.** After the Browser, route, room, and + physical-media checkpoints settle, inventory components, configuration, + migrations, timers, compatibility paths, tests, and truth duplication using + the [maintenance review](./maintenance.md#机制减负审查). Begin read-only and do + not turn it into a broad refactor. -1. **Browser VP8 hardware-acceleration evidence.** After the current physical codec gate, measure supported Chrome/Edge platform combinations under real game load using actual negotiated codec, encoder implementation, power-efficient status, encode time, frame rate, and CPU/GPU attribution. Capability advertisement alone is not proof. Do not add another codec, a custom encoder, GPU-selection workaround, codec ladder, or Native helper without a new accepted decision. -2. **Mobile Viewer lifecycle.** Run Android Chrome and iOS Safari device matrices for autoplay gesture, foreground/background audio, foreground video recovery, lock/page reclamation, rotation, Wi-Fi/cellular migration, and an assigned relay's outbound-media survival or controller failover across those transitions. Preserve the same media element and current reconnect path; do not promise background video composition, background relay continuity, or unsupported lock-screen behavior. -3. **Platform output only when real.** Revisit AirPlay/Cast only when a target browser and physical receiver prove the live `MediaStream` contract. System mirroring remains external; no placeholder button, custom receiver, or server transcode is scheduled. -4. **Public-server one-click deployment package.** After every current functional and real-network item is settled, package the exact application, STUN/SFU, reverse proxy, secrets, and health checks for a user-owned public server. Require real domain/TLS/firewall/capacity inputs and recovery scoped to any infrastructure or irreversible state the installer actually changes; routine application-only immutable updates do not create or maintain a full configuration backup. Never ship credentials or call a partial deployment ready. -5. **Fully local one-click Host-server package.** Package Windows/macOS/Linux Host capture, the exact application server and local state without requiring source, Node or any external Screener/STUN/SFU/tunnel/relay service. Preserve usable direct P2P and use the completed NAT/network evidence to maximize reachability, while reporting public-origin, gateway, NAT and firewall limits truthfully; loopback/LAN success is not universal Internet success. -6. **Native Host capture backend.** At lowest priority, productize a proven Windows native capture path before expanding capture/audio backends to macOS and Linux. Its capture, hardware encode, audio, identity and lifecycle evidence becomes an input to later packages; a source checkout, helper executable, or evaluation ZIP is not a distributable product. -7. **Native shared encode, Windows first.** At lowest priority, prove one hardware encoder output can serve two independent Viewer transports and the Host-SFU publication while each connection retains its own RTP/RTCP, pacing, encryption, feedback, and upload accounting. Productize only after actual access-unit identity, Viewer2/FIFO, heterogeneous feedback, game-resource, browser/hardware, A/V, packaging, and licensing gates pass. Web Host/relay remains standard per-`RTCPeerConnection` encoding because no portable shared-encoder contract exists. -8. **Whole-product UI polish and bilingual-scope decision.** After current functional, physical-media, and real-network work stabilizes, review the product UI once as a whole: simplify and align copy, refine responsive hierarchy and visual consistency, add restrained motion, and measure rendering and bundle cost. Connection presentation uses the two user-facing media labels `P2P` and `SFU` without naming SFU as fallback. A completed reconnect must clear stale “正在重连” stage copy, while retained notices gain an explicit close action. The compact Host roster shows each Viewer's IP for duplicate-name disambiguation without waiting for expanded details; its exact source and data boundary remain part of that task. Decide Chinese/English scope, default/fallback and switching persistence before introducing one shared text catalog; do not add duplicate screen-specific translations or let presentation redefine room and route semantics. -9. **Repository simplification audit.** After the current Browser, physical-media, route, and room checkpoints settle, inventory components, configuration, migrations, gates, timers, compatibility paths, tests, and truth-document duplication using the [maintenance review](./maintenance.md#机制减负审查). Begin read-only: identify the behavior or decision each surface preserves and compare that value with its whole-system cost. Any accepted feature tradeoff updates its single owning contract before complete unused surfaces are removed. This later audit must not become a broad refactor or block current product work. +## Product Decision Needed -## Product Decisions Needed - -| Deployed or retained item | Decision | +| Item | Decision | | --- | --- | -| Project license and closed-commercial boundary | Decide the repository and distribution license before any public release or package distribution. GPL/AGPL implementations remain research-only until then. | +| Repository and distribution license | Decide before public release or package distribution. GPL/AGPL implementations remain research-only until then. | -These decisions are not permission to continue old branches, and current deployment remains unchanged until a scoped decision is made. +## Evidence Boundaries -## Evidence And Later Work - -- The desktop Host background/minimized report is deferred after a bounded current-Chrome screening found no immediate Host-page lifecycle drop and the old `contentHint = "motion"` release remained a plausible confound. Reopen it only with a reproducible current-production real-game case and synchronized capture/send/receive/decode plus CPU/GPU evidence. This does not close or advance the separate Mobile Viewer lifecycle item, whose background playback and assigned-relay survival depend on mobile browser and OS lifecycle behavior. -- Open PR #192 measures synthetic flash/tone timing; it is not an A/V correction feature. -- The Native PR stack #16/#18/#22/#23/#25/#28 and related fanout branches are one research program, not six product TODOs. -- Cap3, resource, route-recovery, Viewer-MBB, signaling-blackhole, Host-generation, SFU-shaping, NAT, mobile, and endurance artifacts remain bounded evidence. Merge or retain them only when a current decision consumes them. Raw-IP ranking, port prediction, NAT classification, AirPlay/Cast receivers, and mobile background guarantees are not implied by the current implementation batch. -- Real heterogeneous-network SFU, mobile lifecycle, audio/A-V device, and endurance/resource acceptance remain owned by the corresponding Now or Later evidence gate. +- Desktop Host background/minimized capture remains diagnostic until a current- + production real-game reproduction correlates capture, outbound, inbound, + decode, and CPU/GPU. Viewer resume correction is not a keepalive guarantee. +- Synthetic and loopback tests validate invariants, not target-network quality, + latency, capacity, mobile lifecycle, or endurance. +- Open Native and packaging branches are retained evidence/candidates, not + current releases. ## Candidate Handling -- Preserve the dirty `fix/configurable-relay-cap` worktree only for user-change audit. Current `main` already implements the accepted uniform `1/2/3` endpoint cap; do not merge the worktree's old route implementation or treat it as unfinished cap work. -- Preserve dirty or unique worktrees and open stacked branches until reviewed. Never resolve their conflicts by importing old truth into `main`. -- Sample worktree, branch, PR, artifact, and reparse-point state immediately before cleanup; do not maintain a permanent workspace inventory here. +- Preserve dirty or unique worktrees and open stacks until reviewed. Never + import old truth wholesale. +- `fix/media-followup` is clean and tree-equivalent to current `main`; it may be + removed only after merged-head, reference, and reparse checks. +- Sample worktree and branch state immediately before cleanup rather than + maintaining a permanent inventory here. diff --git a/docs/verification-status.md b/docs/verification-status.md index 9a7d4f8..69cfa43 100644 --- a/docs/verification-status.md +++ b/docs/verification-status.md @@ -1,56 +1,104 @@ -# Current Verification Status +# Verification Status Last updated: 2026-08-25 -## Purpose +This ledger records evidence that still changes how current source or production +may be interpreted. Git and pull requests own routine completed checks. -This is the demand-loaded ledger for cross-cutting validation evidence and open proof boundaries that do not fit one requirement, ADR, research note, or deployment guide. It is current state, not a release diary: replace superseded facts in place, move durable specialist conclusions to their owning documents, and let Git history retain completed timelines. +## Current Production Identity -Capacity and routing entries below describe the exact tested source or release only. Product direction and executable work are owned by [project memory](./project-memory.md) and [the TODO ledger](./todo.md). +- Production runs exact application/runtime revision + `af348ee1d508a3af02b18a7f46c461953798e19d`, release `af348ee`, wire + `screener-v12`, from `/opt/screener/releases/af348ee`. +- Runtime tar SHA-256: + `4922cc31d7caaad7c5412b1f2fe1d73f788d16d92386efad6eac358a97b8a78e`. + The 39-file manifest SHA-256 is + `5b397809b664b9a55885ac603df615b537bf51276e7274416820870956242d7d`. +- Public Browser asset `assets/index-BvuZY6Hc.js` is 488473 bytes with SHA-256 + `e8fc0500496ee101031a9cfebfcc22d3b69e66d2cac5e875448b56620dbb21c5`. + Public `/healthz` returns 200. Release postflight found Screener, LiveKit, + coturn, and nginx active with zero restarts. -## Deployment Evidence +## Completed Product Evidence -- Production runs exact deployed application/runtime revision `be53c4d5794d38d5d406c876309958d7117ee601`, release `be53c4d`, wire `screener-v12`, from `/opt/screener/releases/be53c4d`; that revision is integrated in canonical `main`. The immutable runtime tar SHA-256 is `5c174c30ffacd98738a364b825b915b84432823c261272684937361be2451144`; its 39-file manifest SHA-256 is `c34dba43a25c6b8447e1153a5be3191bf82cff94117ed31edc325d3f7cf8541f`. -- Postflight found `screener`, LiveKit, coturn, and nginx active with zero restarts and no warning-level Screener/LiveKit journal entries. Local/public health and the served main Browser asset matched the release; `assets/index-BKEVI7Ux.js` has SHA-256 `22114fc8a26b3f03f37973555433d1e73d52d1c7b76fa3c7cc597500823bd62d`, while the prior main asset returns 404. -- Production uses endpoint capacity `2`, room admission `20`, random free four-digit memory rooms, `ROOM_LEASE_SECONDS=86400`, a dedicated LiveKit instance with `room.auto_create: false` and `max_participants: 21`, and global SFU admission `1` ingress / `20` egress. Room authority is process-memory-only. Coturn is `stun-only`, `no-tcp`, and `no-tls`; only UDP 3478 listens, and nft exposes only UDP 3478/7882 for Screener media. -- The production service orders Screener after LiveKit and boundedly waits for its control listener. Media listeners remain STUN-only UDP 3478 and LiveKit UDP 7882 with no 5349, coturn TCP/TLS, relay range, ICE/TCP, or media TCP listener. The stable `inet bonfire_filter` table stayed unchanged through cutover at SHA-256 `afaf9d066e9f292d8bd19032b38c0978ccf60dd01c2fe7f210968ff751f90534`; the whole ruleset is not used as an identity because fail2ban owns a dynamic address set. -- This application-only postflight proves archive identity, atomic application/assets, process health, and listener/firewall continuity. It did not re-exercise access, stale-wire rejection, direct, browser-relay, SFU, audio, codec, mobile, quality, or endurance behavior; those retained results remain revision-bounded below. Routine application-only releases do not maintain a full rollback/configuration backup; any future infrastructure or irreversible-state change requires recovery scoped to what it changes. +- Strict `screener-v12` rejects stale Browser/executable wires before room + authority. Process-memory rooms, 24-hour dormant leases, restart loss, + room-lived Viewer grants, `open | private` code entry, invitation-only private + rooms, password entry, grant rotate/revoke, and `ROOM_NOT_FOUND` have passed + source and production gates. +- Endpoint capacity `1/2/3`, default `2`, one active upstream, acyclicity, + source reachability, deterministic parent order, exact candidate identity, + first-decoded-frame commit, strictly newer rollback, bounded-gap resource + accounting, SFU admission, and delete-plus-absence release have focused unit + and controller coverage. +- Controlled Chrome routing proved direct prepare failure, rollback, SFU + prepare, exact first-frame ready, SFU commit, and continuing UDP media without + external ICE servers on Browser LiveKit PCs. This does not replace public + heterogeneous-network recovery evidence. +- Chrome direct and Browser-relay VP8 measurements reached about 60 fps after + stock bandwidth-estimation warm-up. Pinned LiveKit client `2.22.0` and server + `1.13.5` separately proved native lower/higher screen-share representation + selection with Dynacast and server send-side BWE. +- Direct, Browser-relay, and SFU paths advanced video plus screen audio at + 64/128/256 kbps ceilings and after source replacement. The old constant + active-audio `1 kbps` display is closed. Exact SFU weak-network behavior after + RED was disabled remains open. +- Browser SFU publisher/subscriber PCs use empty external ICE-server lists while + ordinary peers retain deployment STUN. The reported Host/Viewer TUN conflict + was closed by that isolation without adding TURN or TCP media. -## Retained Functional Evidence +## Current Source Evidence -- Exact deployed source `be53c4d` passed repository hygiene, TypeScript typecheck, all 613 Web tests in 44 files, client and server production builds, two independent reviews, pinned LiveKit encoding calculation, and production artifact/service postflight. Exact predecessor `b68c471` separately passed the 261-test SFU admission gate and explicit-Chrome access/privacy gate. Retained Chrome 20-Viewer and responsive visual checks apply only to their recorded predecessor revisions. -- Chrome 151 on exact production `1d87615` used real dynamic-tab `getDisplayMedia()` at 1822x1080 and made only the ordinary STUN peer attempt unusable so the existing route operation reached its SFU suffix. Publisher and subscriber LiveKit PCs retained empty external ICE-server lists and selected UDP. The publisher exposed one video sender encoding and one outbound video stats object, both without RID; over six seconds it encoded 180 frames while the subscriber decoded and rendered 182 frames, with bytes advancing at both ends. This closes the single-representation implementation/deployment gate without claiming public SFU 60 fps, real-game quality, heterogeneous-network behavior, or endurance. -- Chrome 151 on exact production `1d87615` used one continuously changing synthetic Browser stream while test-side policy made only ordinary STUN peer PCs unusable and left LiveKit unchanged. The Viewer observed direct prepare revision `1`, explicit rollback active revision `2`, SFU prepare revision `3`, exact-revision first-frame ready, then SFU active revision `3`; no ready was sent for the failed direct candidate. The final Host snapshot had no operation, parent/final route `sfu`, rejection `none`, and first-frame/final timing `10703/10703` ms within the unchanged 20-second total deadline. Host and Viewer LiveKit PCs had zero external ICE URLs, selected UDP, and advanced `58/57` frames plus bytes over the next two seconds. This closes the exact deployed controlled candidate rollback/SFU commit transaction, not physical capture, heterogeneous-network reachability, all-UDP-blocked failure, quality, or endurance. -- Chrome 151 on exact production `1d87615` formed `Host -> SFU -> relay -> child` plus a separate direct Viewer, passed Pause/Resume, and then locally closed the relay's subscriber PC. After 45,234 ms the relay remained assigned to SFU and sent no `route-media-unavailable`, `refresh-sfu`, or `route-failed`; its downstream child independently reported its own decoded stall and recovered through the other Viewer. Receiver-stats calls had stopped about 28.9 seconds before the final sample. Source inspection confirms that no video track stops SFU sampling while an empty merged report or failed stats read produces no deadline sample. This bounded fault injection exposes an active-cadence implementation gap; local `RTCPeerConnection.close()` does not reproduce physical UDP loss or prove LiveKit's real reconnect path, so relay-ingress recovery remains open. -- Chrome 151 on exact production `bf32859` captured a continuously changing tab with generated audio through real `getDisplayMedia()`. Two Host-direct children plus one browser-relay child and one forced-SFU Viewer all advanced decoded audio energy and video frames at 64/128/256 kbps sender ceilings and after live source replacement. Ordinary paths selected direct UDP; the SFU publisher/subscriber selected LiveKit UDP with empty external ICE-server lists. The UI never reported the faulty constant `1 kbps` with active audio. Direct/browser-relay actual traffic tracked each selected ceiling; SFU 64/128 traffic was higher because the accepted LiveKit default retains RED, so sender ceiling/readback and wire traffic remain distinct. After RTCP mapping became available, direct/browser-relay audio-minus-video playout timestamps ranged from -101 to +4 ms; Chrome exposed no corresponding value on the SFU subscription, so real-game SFU A/V synchronization remains open rather than inferred. -- Chrome 151 on exact production `bf32859` used real `getDisplayMedia()`, a continuously changing source, separate Browser processes, VP8/libvpx, and no video hint. Direct stabilized near 59.5 encoded/received/decoded/rendered fps at 1904x928; a Browser relay kept every stage near 59.4--59.7 fps, adapting its outbound to 1428x696 under bandwidth pressure. Both converged after about 25--30 seconds of stock BWE ramp-up. On the exact public SFU path, the then-current `q,h` publication held `h` near 9.2 fps/1.37 Mbps; making `q` inactive from the first sender-parameter application raised the same 1904x928 `h` to about 23.3 fps/3.32 Mbps with source/capture near 60, 4.46 ms encode time per frame, zero loss/retransmission, and about 4.81 Mbps available outgoing bitrate. That gate closed an intrinsic VP8/no-hint direct/relay shortfall and rejected always-active `LOW`; it did not prove a deployed single-representation SFU path, public SFU 60 fps, or real-game performance. -- Exact Browser SFU ICE-server-isolation implementation `ae09c760adec76fd26da611d4928486d105c6d3b` passed repository hygiene, TypeScript typecheck, all 605 Web tests in 45 files, client and server production builds, three independent code reviews, focused 95-test publisher/subscriber and ordinary-peer isolation coverage, deployment postflight, and owner physical SFU fallback media proof. This establishes the reported production connection fix without turning candidate type/count into an invariant; full route/network acceptance remains open. -- A fresh canonical `main@c4962f5` background gate passed repository hygiene, all 605 Web tests, both production builds, the 254-test SFU admission suite, and the Chrome 151 20-Viewer 720p30/30-second local route gate. All 20 Viewers advanced decoded frames, Host and relays stayed within endpoint cap `2`, no orphan SFU publication appeared, and all four current-child timing distributions contained 20 values. The browser gate uses deterministic canvas and does not start LiveKit, so it proves local direct/browser-relay control and media behavior only; it is not physical capture or SFU evidence. -- Exact v10 runtime source `fdd5a4a529ff297f41c05ea3388bf484d76afe8f` passed repository hygiene, TypeScript typecheck, client and server production builds, all 610 Web tests in 45 files, the 257-test SFU admission gate, the access/privacy Browser gate, and the production-dependency audit with zero vulnerabilities. Its one-Host/20-Viewer Chrome 151 direct-loopback report passed all seven assertions: all Viewers decoded, the selected VP8 settings reached every participant, Host and relay media edges stayed within cap `2`, no orphan SFU publication appeared, and all four current-child timing distributions contained 20 values. The report SHA-256 is `802CBB7632E9AEA3C39FE831A7F7880D24140B2E83942CE0DFDA05DE4B408EED`. This validates the v10 source boundary, not H.264 quality, hardware attribution, physical capture, public-network behavior, or SFU packet flow. -- Chrome 151 synthetic `1/3/5/8` and 720p30 runs preserved bounded fanout and decoding. A `9461e20`-era three-Viewer rerun kept Host2/Browser Viewer1, decoded every Viewer, and recovered a hard first-level relay departure in 5,380 ms; it did not exercise quality-triggered MBB. Sixteen-Viewer cap2/cap3 loopbacks decoded all Viewers and a ten-second follow-up recorded guarded sender and aggregate Chromium CPU evidence, but resolution/FPS remained low and the ordinary-PC data cannot rank caps, select product policy, or prove sustainable heterogeneous-network quality. See [peer-assisted media](./research/peer-assisted-media.md). -- Chrome 151 with local LiveKit 1.13.5/client 2.22.0 proved one SFU/UDP root after `q,f` -> `q,h`: decoded frames advanced 3 -> 23, rendered frames reached 26, Host used one edge, and leaves were clean. This is functional evidence only. See [low-server media routes](./research/low-server-media-routes.md). -- Chrome 151 on the current Windows TUN path reproduced intermittent production LiveKit initial ICE before publication: server counters observed both ingress and egress while Browser ICE received no responses. Eight isolated Host attempts using the LiveKit-provided STUN endpoint succeeded `2/8`; explicitly empty Browser SFU `iceServers` succeeded `8/8`, observed a peer-reflexive-to-host UDP pair within 0.5--1.7 seconds, and published 75--89 ms later. This selects Browser SFU ICE-server isolation; the observed candidate type is diagnostic, not a gate, and the run does not close exact deployed Host/Viewer first-frame, mobile, quality, or endurance evidence. See [low-server media routes](./research/low-server-media-routes.md). -- Host A+B and authenticated P2P Viewer C evidence is sanitized, generation-bound, diagnostic-only, and closed for stale or ambiguous reports. Production exposes one Host-local current SFU sender snapshot and clears it on publisher identity changes; raw media metadata is not retained. See [realtime quality adaptation](./research/realtime-quality-adaptation.md). -- Fresh Chrome 151 tab-capture probes identified two separate Browser boundaries. Removing video `contentHint = "motion"` changed VP8 from 14.93 fps at 428x208 to 29.73 fps at 1904x928 under the same 5 Mbps balanced setup. No-hint AMD H.264 remained about 14--16 fps with hundreds of outer rate-limiter drops, while the same MFT reached 29.93 fps under diagnostic Desktop SW BRC; OpenH264 was slower. The page cannot enable that Chromium field trial or select an adapter/MFT. No-hint VP8 used libvpx software and held about 30 fps in one- and two-sender loopback on the test machine, but the process CPU totals include capture and local decode and are not real-game or weaker-device evidence. This evidence selects fixed VP8 for the Browser contract; current source and production implement that decision, and exact-production gates verify VP8 on direct, Browser-relay, and single-representation SFU paths. See [realtime quality adaptation](./research/realtime-quality-adaptation.md), [Native H.264](./research/native-h264-hardware-decision.md), and [Native shared encode](./research/native-shared-encode-sender.md). -- One bounded Chrome 151/Windows 25H2 current-browser screening used real `getDisplayMedia()` window capture, VP8/libvpx with no video hint, a continuously changing WPF source, and one same-machine loopback receiver. Focused, hidden-tab, and minimized 15-second windows measured about 29.4--29.6 capture/encoded/decoded fps and 1.35--1.50 ms encode time per frame; each boundary sample read an enabled, unmuted, live 1186x712 track and each end sample reported `qualityLimitationReason=none`. All rates used monotonic counter deltas over the outbound-RTP RTCStats timestamp interval; source/inbound timestamps and raw stats identities were not retained independently. The sample did not observe an immediate Host-document background drop; it does not cover exact production code, a real game, physical Viewer, CPU/GPU contention, audio, repetition, a hold beyond five minutes, or public routing. See [Browser background capture](./research/browser-background-capture.md). -- Native Windows 11 process audio is source-only and default-off. A retained run isolated the target by about 4018x and delivered 495 Opus packets to one Viewer; package download, game sync, Windows 10, and other routes remain open. See [ADR-0008](./adr/0008-window-scoped-audio-capture.md). +- `af348ee` fixes Browser VP8 with `contentHint = "motion"`, reapplies the + selected video profile after answer negotiation, leaves SFU representation + construction to pinned LiveKit, keeps Dynacast/send-side BWE, disables + AdaptiveStream, and disables SFU audio RED. +- The accepted `fix/viewer-page-lifecycle` candidate has focused coverage for + page-resume decoded-stall rebaselining, current-frame presentation authority, + and same-parent P2P manual rebuild. TypeScript and the client production build + pass. It is not current source or production until its release is integrated. -## Open Proof Boundaries +## Open Physical Gates -- Routing: exact v10 local direct loopback remains historical evidence for admission-20 initial ingress. Exact v12 production closes one controlled ordinary-direct exhaustion, explicit rollback, and exact SFU first-frame commit transaction; the later subscriber-PC closure injection found the active-SFU cadence gap without proving physical loss recovery. Current source corrects that cadence while physical recovery remains open. Representative real-network direct/peer/SFU behavior, relay-ingress reparent with subtree retention, disconnect/effective-capacity drain, pause/resume, and bounded all-UDP-blocked failure remain open. Silent control partitions may wait 30 to 60 seconds for heartbeat detection before the default five-second grace; a non-paused decoded-frame stall may invalidate only that exact child edge. -- Quality: exact-production direct and Browser-relay VP8/no-hint cadence plus the historical single-encoding SFU baseline are closed. ADR-0007's pinned gate separately proved LiveKit-owned selection between full- and half-resolution representations with server send-side BWE, and production now caps the half-actual-resolution lower representation at 30 fps with proportional bitrate. Its mixed-route HIGH recovery, constrained LOW delivery on this release, public SFU 60 fps, real-game quality, heterogeneous devices, and endurance remain open. Startup ramp is stock BWE behavior and does not authorize an SDP or application bitrate workaround. -- Host background: deferred until the report reproduces on exact production with a continuously changing real game or replay, a separate direct wired Viewer, synchronized capture/outbound/inbound plus CPU/GPU evidence, and repeated focused/unfocused/hidden/minimized windows. The current-Chrome Browser `getDisplayMedia()` WPF-window screening did not observe an immediate Host-page media drop, and the old harmful `contentHint = "motion"` release remains a plausible confound rather than a proven background cause. Current evidence authorizes no keepalive, Wake Lock, timer, codec switch, route change, or Native fallback. -- Relay resources: current source and production expose the authenticated `1/2/3` endpoint-cap boundary, and retained v10 local loopback evidence covers 20 Viewers at cap `2` only for that source. CPU/GPU/upload and heterogeneous-network measurements remain diagnostics, not authority to change the contract. -- Room scale: current source and production admit 20 Viewers plus one Host, and the retained v10 local Browser convergence gate covers that bound on direct loopback. A real public-network 1:20 session, quality, and sustained resource matrix remain open. -- SFU: current source and production use one half-actual-resolution lower representation capped at 30 fps plus the original representation, Dynacast, and the tracked server send-side-BWE setting without an application layer selector. Both use exact Viewer subscriptions and no fixed root-count or room-wide lease rule. Chrome 151 physically closed the empty-external-ICE-server connection hold and historical single-representation gate, while the separate pinned-stack gate proved native representation selection. The lighter lower representation's production quality effect, real LiveKit lifecycle/JWT replay, resource use, SFU-fed first-child behavior, and broader heterogeneous-network acceptance remain open. -- Audio: current v12 source and production select the exact active ordinary Host or SFU publisher audio sender, reset samples across track changes or absence, keep relay children synchronized with persistent-stream audio changes, and reacquire exact SFU publications/senders after reconnect. Chrome 151 on exact production `bf32859` physically closed the reported active-source `1 kbps` observation and all three live ceilings across direct, browser-relay and SFU. Audible game quality, heterogeneous-network changes and real-game A/V synchronization remain part of route acceptance; a muted/idle Opus track may still legitimately remain near 1 kbps. -- Observability: current v12 source and production keep local connection details but no Host/Viewer diagnostic downloads or Browser file-export plumbing. The Host-only bounded snapshot stays for automatic acceptance. Chrome 151 supplied selected candidate endpoints and ordinary-route A/V playout timestamps in the current gate, but omitted the SFU A/V timestamp. Target-browser availability and actual values remain open for codec/fmtp and encoder implementation/power efficiency, media-source and encoded FPS, and broader jitter/concealment cases. All retained fields diagnose and have no recovery authority; they do not by themselves prove hardware encode, synchronization quality, or route quality. -- Mobile: Android/iOS remain Viewer-only. Autoplay, rotation, iOS lock-screen/page reclamation, background reconnection, and network changes need device evidence. AirPlay/system mirroring is Viewer-local output, not a portable in-app WebRTC output contract. -- Native: ADR-0006 has one-Viewer evidence only. Viewer2/FIFO, hardware diversity, endurance, downloaded-package/public-host, game A/V, and browser diversity remain open. -- Access and operations: exact predecessor `b68c471` passed the access/privacy gate with explicit Chrome and production cutover/postflight, including stale-v11 rejection before authority, room-lived grant/code orthogonality, private entry with and without a password, rotate/revoke, missing-room typing, and memory-only runtime. Headful same-browser preference replay and the isolated VM/IP clean-port matrix remain open; shared-IP smoke cannot approve them. +### Route And Network -## Evidence Rules +- Repeat direct, peer-relay, SFU, relay-ingress recovery with subtree retention, + disconnect/capacity drain, and Pause/Resume on representative IPv4/IPv6, + Wi-Fi/cellular, and Host/Viewer TUN/VPN networks. +- Verify all-UDP-blocked networks reach bounded explicit failure. No TURN, + ICE/TCP, media TCP, TLS relay, NAT classification, port prediction, or TCP + probe is authorized. +- Run the 20-Viewer public-network capacity/resource/endurance gate. Local or + synthetic 1:20 control evidence does not establish target-network quality. -- Target-device and production measurements drive performance decisions. Ordinary-PC synthetic runs prove only correctness or interoperability. -- Record expensive evidence with exact source, environment, result, and applicability. When a relevant media path changes, replace or invalidate the corresponding entry; unrelated documentation changes do not require rerunning it. -- Keep architecture-splitting candidates out of this ledger unless a proven consumer boundary makes them active work; file length alone is not evidence. +### Media Quality + +- Measure real games at 720p30, 1080p30, and 1080p60 across weaker Hosts and + actual Browser encoder implementations. Correlate capture, outbound, inbound, + decode, CPU/GPU, and A/V timing. +- Verify pinned LiveKit default representation behavior on exact production + under constrained and unconstrained subscribers without adding an application + layer selector. +- Verify weak-network screen audio and A/V synchronization after RED was + disabled, including concealment/FEC evidence where exposed. + +### Lifecycle + +- Desktop Host background/minimized capture remains unconfirmed. A short + current-Chrome window-capture screening did not reproduce an immediate drop; + a current-production real-game reproduction must separate Host page, captured + surface, capture, encoder, path, and Viewer behavior. +- Android Chrome and iOS Safari still need autoplay, background audio, + foreground video recovery, page reclamation, rotation, network migration, and + assigned-relay survival tests. Web does not promise background video or relay + execution after OS suspension. + +## Interpretation Rules + +- Configuration and loopback prove neither target-network quality nor hardware + acceleration. +- Missing RTCStats fields are unknown, not zero. +- Current-path quality is diagnostic and cannot authorize active parent change. +- A Browser page lifecycle correction is recovery logic, not keepalive. +- Native, packaging, and broad UI work remain outside these gates until their + TODO decisions are opened. diff --git a/docs/方案设计.md b/docs/方案设计.md index 169815f..125d12d 100644 --- a/docs/方案设计.md +++ b/docs/方案设计.md @@ -2,7 +2,7 @@ ## 2026-08-18 14:45 -状态：production 运行精确 application/runtime revision `1d8761528d0dba43fb6d818df3934483ba2f5340`、release `1d87615` 和严格 `screener-v12`；canonical `main` 含相同 application/runtime tree 与当前 truth checkpoint。源码与生产已实现 22 字符 room-lived Viewer grant、`ROOM_NOT_FOUND`、产品诊断下载移除、完整短 URL 邀请 UI、收敛的房间密码控件、exact-current audio stats 与音轨跨 relay/SFU reconnect 连续性，并固定 Browser VP8，video 不设置 `contentHint`、audio 保留 `music`，且无 codec UI/state/wire 或分享生命周期分支；Pause/Resume 继续由当前 Host session 和 `shareGeneration` 约束。Browser SFU 发布为单个无 RID 的 `HIGH` encoding，不启用 simulcast 或 Dynacast；Chrome 151 已在该精确 production 上验证发布端与订阅端经 LiveKit UDP 持续推进帧。 +状态：production 与 canonical `main` 运行精确 application/runtime revision `af348ee1d508a3af02b18a7f46c461953798e19d`、release `af348ee` 和严格 `screener-v12`。Browser 视频固定 VP8 并使用 `contentHint = "motion"`，音频使用 `music`；codec UI/state/wire、自定义 SFU layer list 与应用媒体质量 controller 均不存在。SFU 由 pinned LiveKit 默认 representations、Dynacast、server send-side BWE 和 disabled AdaptiveStream 拥有质量适配；screen audio 使用 64/128/256 kbps ceiling、stereo、DTX off 与 RED off。当前 Viewer 页面生命周期/原生控件候选已接受但尚未集成或部署，精确执行状态由 `docs/status.md` 持有。 ## 目标与边界 @@ -158,7 +158,7 @@ accepted 的唯一 wire 是 `screener-v12`，不做版本协商、旧 parser、 | 1080p30 | 1920x1080、30 fps | 5 Mbps/peer | | 720p30 | 1280x720、30 fps | 3 Mbps/peer | -浏览器可返回更低设置，界面以 `track.getSettings()` 显示实际捕获结果。推荐区始终只有表中的三档并默认选择中档 `1080p30`；高级分辨率额外提供 `480p`（854x480）、720p、1080p 与 1440p，FPS 继续独立接受 15 至 60 整数，码率继续独立接受 2 至 12 Mbps，因此不建立第四个推荐档或 `480p` preset ID。视频 track 不赋任何 `contentHint`，让 display capture 保留浏览器的 screen 分类；返回的音频 track 使用 `contentHint = "music"`。三档推荐质量与折叠高级区初始值统一采用 `degradationPreference = "balanced"`，并允许显式选择 clarity-first 或 fluid。“分享高级设置”不提供 codec 选择，并提供离散的 `64 kbps 省流 | 128 kbps 音乐默认 | 256 kbps 极高音质` 屏幕音频 sender ceiling。音频 ceiling 可在直播中串行更新。这些设置只是 ceiling 和浏览器偏好，不保证实际分辨率、帧率、码率或声道。 +浏览器可返回更低设置，界面以 `track.getSettings()` 显示实际捕获结果。推荐区始终只有表中的三档并默认选择中档 `1080p30`；高级分辨率额外提供 `480p`（854x480）、720p、1080p 与 1440p，FPS 继续独立接受 15 至 60 整数，码率继续独立接受 2 至 12 Mbps，因此不建立第四个推荐档或 `480p` preset ID。视频 track 使用标准 `contentHint = "motion"` 表达游戏动态意图；返回的音频 track 使用 `contentHint = "music"`。三档推荐质量与折叠高级区初始值统一采用 `degradationPreference = "balanced"`，并允许显式选择 clarity-first 或 fluid。“分享高级设置”不提供 codec 选择，并提供离散的 `64 kbps 省流 | 128 kbps 音乐默认 | 256 kbps 极高音质` 屏幕音频 sender ceiling。音频 ceiling 可在直播中串行更新。这些设置只是 ceiling 和浏览器偏好，不保证实际分辨率、帧率、码率或声道。 每次 `setParameters()` 都必须从当次 `getParameters()` 派生更新，然后读回 requested/applied `maxBitrate`、`maxFramerate`、`scaleResolutionDownBy` 和 degradation preference。浏览器负责具体降级，实际结果由 stats 观测；拒绝或浏览器改写必须进入可见状态，不能只写 `console.warn`。 @@ -176,19 +176,19 @@ Web display-audio 请求显式关闭 `echoCancellation`、`noiseSuppression`、` Host 本地预览直接绑定现有 capture stream，不创建 viewer、`RTCPeerConnection` 或额外媒体边，也不产生 SFU/应用服务器出口。预览仍可能有本地合成、缩放和 GPU 显示成本；在同一动态场景对预览开/关采集 Host CPU、GPU、encode time 和实际 FPS。后台性能诊断必须把 Host 页面前台/遮挡/最小化、被捕获 source 前台/遮挡/最小化以及 preview 播放/暂停分成独立变量，并关联 document visibility/focus、track mute/readyState 与 capture/outbound/inbound stats。页面 timer、伪活跃和 Screen Wake Lock 都不构成屏幕捕获保活方案；若标准浏览器无法维持目标行为，应记录平台能力边界而不是制造第二套 lifecycle。只有稳定收益且捕获、编码和远端画面不变时，才把直播后的默认动态预览改为静止缩略图或隐藏并允许按需展开；隐藏预览绝不停止 capture track 或 sender encode。 -codec 与 content-hint 依据由研究文档持有。`motion` 会把 Chromium/libwebrtc 的 display capture 从 screen 改成 realtime，并在同一 VP8 条件下造成明显降帧和降分辨率，因此产品不设置视频 hint。无 hint VP8 在当前受控证据中恢复接近 30 fps、完整分辨率和正常码控，具备 Browser 基线覆盖；它因此成为唯一 Browser codec。普通 AMD H.264 MFT 的浏览器码控仍丢帧，而诊断性 Desktop SW BRC 可恢复约 30 fps，说明问题不等于硬件编码器本身的吞吐上限；网页没有启用该浏览器进程开关或选择具体 GPU/MFT 的 API。实际 codec 与 encoder 字段只用于诊断，不触发 codec 切换。VP8 hardware acceleration 需要后续按目标浏览器和真实游戏负载单独取证；本设计不假定其存在，也不增加替代 codec 或自定义编码路径。 +codec 与 content-hint 依据由研究文档持有。`motion` 会改变 Chromium/libwebrtc 的适配取舍并可能空间降级以保留游戏动态；它是标准内容意图，不是分辨率或帧率保证。产品不再用 unset hint 固定 screen-classified 行为，也不围绕某次测量实现应用缩放器。普通 AMD H.264 MFT 的浏览器码控仍丢帧，而诊断性 Desktop SW BRC 可恢复约 30 fps，说明问题不等于硬件编码器本身的吞吐上限；网页没有启用该浏览器进程开关或选择具体 GPU/MFT 的 API。实际 codec 与 encoder 字段只用于诊断，不触发 codec 切换。VP8 hardware acceleration 需要后续按目标浏览器和真实游戏负载单独取证；本设计不假定其存在，也不增加替代 codec 或自定义编码路径。 `track.getSettings()` 中存在的 EC/NS/AGC/voice-isolation、`channelCount` 和 `sampleRate` 只能作为当前捕获观测；缺失字段保持未知，Opus 协商仍是独立事实。每个 Web Host 或浏览器 relay 的实际音频 sender 在首次绑定和换源后通过标准 `RTCRtpSender.setParameters()` 请求并读回当前 64/128/256 kbps ceiling；这只是上限，浏览器 WebRTC 拥塞控制仍可降低实际流量，不新增应用层音频降级器。128 kbps 保持默认，因为 RFC 7587 将 20 ms full-band stereo music 的建议区间列为 64--128 kbps，pinned LiveKit 2.22.0 也把 128 kbps 命名为 `musicHighQualityStereo`；256 kbps 是用户显式选择的较高上限，不是质量保证。Peer answer 声明 `stereo=1;maxaveragebitrate=256000`，让同一接收协商覆盖所有三档，实际 sender ceiling 仍由所选档位约束；普通 PC synthetic 不用于标定参数、听感或性能。 Web peer 的最小 stereo 契约只改 Viewer 生成的 answer。实现把 MIT `sdp-transform` 2.15.0 与 `@types/sdp-transform` 声明为 root direct dependencies，不依赖 LiveKit 的 transitive install，也不用字符串或正则拼接 SDP。Helper 仅在解析出唯一非 rejected audio media、通过结构化 `rtp` entry 找到唯一 Opus payload 时，才在该 payload 的 fmtp 幂等 upsert `stereo=1` 与 `maxaveragebitrate=256000`；随后 `ViewerPeer` 把同一个结果交给 `setLocalDescription()` 和 signaling。首次连接、ICE restart、同 parent rebuild与浏览器 relay child 都走该边界。空 SDP、无或歧义 Opus、多个有效 audio、重复目标参数、畸形 fmtp、parse/write 异常都按对象 identity 返回浏览器原 answer，保住仍可继续的 mono/video edge，不发送半修改 SDP。RFC 7587 说明这两个值都是可选的单向 receiver parameters；`maxaveragebitrate` 是 maximum 而非最低保证，实际发送仍可随拥塞下降。当前 libwebrtc 把缺省 full-band stereo codec bitrate 配成 64 kbps，而 `RTCRtpSender.maxBitrate` 只限制上界；本接收上限允许三档，实际发送预算仍由 64/128/256 kbps sender ceiling 与拥塞共同决定。`sprop-stereo` 只是 sender likelihood hint，`usedtx` 缺省已是 0，本 slice 不增加二者，也不改浏览器协商的 `useinbandfec`。 -SFU publisher 将 64 kbps 映射到 `AudioPresets.musicStereo`、128 kbps 映射到 `musicHighQualityStereo`，256 kbps 使用显式 `maxBitrate`；三者都保留 `forceStereo: true` 与 `dtx: false`。LiveKit 2.22.0 先合并 room publish defaults，其中 `red: true`；本实现不传 `red` 覆盖值，保留既有 RED request。RED 与 Opus in-band FEC 是不同修复机制，保留前者不声称实际启用、替代或等价于后者。直播中只改变档位时，不重新协商 Opus 或 republish；服务端沿现有 quality wire 保存 last-wins desired profile，各客户端 sender-mutation queue 从 fresh parameters 更新当前 P2P、relay 和 SFU audio sender并立即 readback，过期 completion 不覆盖新值，未来 sender 读取最新 desired profile。应用失败保留该 sender 的旧 applied 值和媒体并显示告警；无远端 applied 回执时 Host 不宣称全房原子收敛。Pinned LiveKit 2.22.0 的当前 Chrome/Edge screen-audio baseline 可直接更新现存 audio sender；其他浏览器以实际 readback 为准。Viewer 继续在同一个持久 `MediaStream`/媒体元素中增删 screen video/audio track；不新建 audio element、Web Audio mix/resample graph、第二 audio representation 或独立音频时钟。默认每条 outbound edge 的 audio RTP payload ceiling 为 128 kbps，显式最高档为 256 kbps；非服务器端点容量为 C 时最多 C × 128/C × 256 kbps（不含 RTP/SRTP/UDP/IP overhead）。stereo 仍可能增加编码与网络成本，且浏览器 relay 不保证跨 PeerConnection shared encode。未来 microphone voice 若进入范围，另建独立 source/track 与 AEC、降噪、gain、DTX 设计，不能作用于电影/游戏屏幕音频。详细证据与 Oopz 未知边界见 `docs/research/browser-screen-audio-quality.md`。 +SFU publisher 将 64 kbps 映射到 `AudioPresets.musicStereo`、128 kbps 映射到 `musicHighQualityStereo`，256 kbps 使用显式 `maxBitrate`；三者都使用 `forceStereo: true`、`dtx: false` 与 `red: false`。RED 与 Opus in-band FEC 是不同修复机制；关闭 RED 降低冗余流量并接受其弱网恢复退让，不声称禁用或替代 Opus FEC。直播中只改变档位时，不重新协商 Opus 或 republish；服务端沿现有 quality wire 保存 last-wins desired profile，各客户端 sender-mutation queue 从 fresh parameters 更新当前 P2P、relay 和 SFU audio sender并立即 readback，过期 completion 不覆盖新值，未来 sender 读取最新 desired profile。应用失败保留该 sender 的旧 applied 值和媒体并显示告警；无远端 applied 回执时 Host 不宣称全房原子收敛。Viewer 继续在同一个持久 `MediaStream`/媒体元素中增删 screen video/audio track；不新建 audio element、Web Audio mix/resample graph、第二 audio representation 或独立音频时钟。详细证据见 `docs/research/browser-screen-audio-quality.md`。 Viewer 本地连接详情只从当前 route 的唯一 inbound screen video/audio stats 读取同步证据：`audio estimatedPlayoutTimestamp - video estimatedPlayoutTimestamp` 表示当前发送端时间线差；音视频 jitter-buffer delay 取相邻窗口累计 delay/count 的比值；音频 concealment 取相邻窗口 concealed/total sample 与 event 增量。每个媒体种类用 stats id、SSRC 和 track identifier 共同守住区间身份，换流或累计计数回退时先重新基线，字段缺失或分母为零时保持未知。该证据不进入 signaling、质量上报或持久化，也不设置 `jitterBufferTarget`、Web Audio 或任何媒体控制参数；真实同步仍由同一持久 `MediaStream`/媒体元素和 WebRTC playout 负责。 `frameRate` 约束和 sender 的 `maxFramerate` 都是目标或上限；浏览器仍控制实际捕获、重复帧处理、编码和拥塞降级。静态画面可能由浏览器减少实际编码数据，但标准没有保证每个实现以同一种方式节省采集或 GPU 开销。本 PoC 不添加画面差分、定时改约束或自定义动态 FPS 状态机，先用 `framesEncoded`、实际发送码率、`totalEncodeTime`、质量受限原因和主机资源占用测量真实问题。 -质量自适应采用 ADR-0007 的路径隔离框架能力，而不是全房间按最差链路降档。Direct/peer 保留每条 PeerConnection 的 stock WebRTC GCC。Browser Host 的唯一 SFU publication 不手写层列表，由锁定的 LiveKit client 以半分辨率和原始分辨率两路 VP8 screen-share encoding 提供空间表示。LiveKit Dynacast 按全房最高需求停用其上的发布层，无订阅时停用全部层；启用 send-side BWE 的 server stream allocator 独立选择每个 subscriber 的实际下行层，订阅端最高质量请求只是上限。Screener 不实现质量评分、带宽估计、周期调档或应用媒体层 selector。当前任一 Viewer 都可能承担 peer children，因此 subscriber 统一保持 `adaptiveStream: false`，relay ingress 不跟随本地 DOM 可见性。Pinned gate 已证明受限下游原生选择半分辨率、非受限下游取得原始分辨率、手动及元素驱动的同连接恢复和 Dynacast 启停；也证明最高需求会让两路同时编码。双编码是支持受限下游的固定成本，不是回退到单路的理由。当前源码已完成迁移，生产仍是单路 encoding。 +质量自适应采用 ADR-0007 的框架所有权，而不是全房间按最差链路降档。Direct/peer 保留每条 PeerConnection 的 stock WebRTC GCC。Browser Host 的唯一 SFU publication 不手写层列表，由锁定的 LiveKit client 默认行为拥有 VP8 screen-share representations、source replacement 与 republish。LiveKit Dynacast 和启用 send-side BWE 的 server stream allocator 独立处理聚合需求与每个 subscriber 的实际下行层，订阅端最高质量请求只是上限。Screener 不实现质量评分、带宽估计、周期调档或应用媒体层 selector。当前任一 Viewer 都可能承担 peer children，因此 subscriber 统一保持 `adaptiveStream: false`，relay ingress 不跟随本地 DOM 可见性。Pinned gate 证明了 LiveKit 原生空间选择和 Dynacast 能力，但具体 representation 参数仍属于 pinned component，而不是 Screener 产品常量。 质量证据只用于诊断，不形成全房分数或 parent authority。路由只消费 exact child edge 的 `failed/closed` 和非暂停 decoded-frame stall。prepare/commit/abort 绑定 authenticated sessions、share generation、base/pending revision 与 candidate connection；candidate child 的第一帧新 decoded video 是唯一应用层 media-ready，ICE connected 不能单独提交。 @@ -206,7 +206,7 @@ Viewer 本地连接详情只从当前 route 的唯一 inbound screen video/audio Viewer 的已鉴权质量证据只描述当前路径，不能命令 WebRTC/LiveKit 改层、改变 Host publication 或改变拓扑。UA、平台和设备型号不参与质量或 relay capacity 决策。 -SVC 的标准能力门禁已经静态关闭为 `no-go-web-svc-cross-path-hardware-contract`。WebRTC-SVC 只配置单个 outgoing sender，direct/peer receiver 没有对应的按层选择 API，独立 PeerConnection 之间也没有 shared-encode 契约；requested/applied codec/`scalabilityMode`、Media Capabilities `powerEfficient` 与 RTCStats encoder 字段不能证明编码器一定是硬件。Pinned LiveKit 虽能在 SFU 内为每个 subscriber 设置最大空间/时间层请求，但实际选择依 codec/selector 而异：server 1.13.5 只为 VP8 安装 temporal selector，当前 H.264 `q,h` simulcast 只有空间层选择。Client 2.22.0 还会把 screen-share SVC 强制成三层 temporal-only `L1T3`，无法提供当前 Browser 合同要求的 LOW/HIGH 两种空间表示；这种 SFU 选层也不延伸到 direct/peer descendants。未来只能由能明确锁定硬编、最多两层、跨 direct/peer/SFU 复用同一输出且通过游戏性能/延迟门槛的 native sender 以新决策重开。普通非分层编码包不能只靠转发生成另一档画质，实现路径差异只能用第二表示、合格的 scalable layers 或 relay/SFU 转码。 +SVC 的标准能力门禁已经静态关闭为 `no-go-web-svc-cross-path-hardware-contract`。WebRTC-SVC 只配置单个 outgoing sender，direct/peer receiver 没有对应的按层选择 API，独立 PeerConnection 之间也没有 shared-encode 契约；requested/applied codec/`scalabilityMode`、Media Capabilities `powerEfficient` 与 RTCStats encoder 字段不能证明编码器一定是硬件。Pinned LiveKit 可在 SFU 内为每个 subscriber 设置最大空间/时间层请求，但实际选择依 codec、SDK 默认和 server selector 而异，且不延伸到 direct/peer descendants。未来只能由能明确锁定硬编、跨 direct/peer/SFU 复用合格输出且通过游戏性能/延迟门槛的 native sender 以新决策重开。普通非分层编码包不能只靠转发生成另一档画质，实现路径差异只能由届时接受的 scalable representation 或转码模型拥有。 质量控制不放宽稳态 outbound media-copy 预算：所有非服务器端点继续受统一静态 capacity 约束；native relay 只转发选定表示的编码包，不解码重编码。后续双树或条带化必须另行验收，不能绕过当前 graph、identity 和 capacity invariant。 @@ -295,15 +295,15 @@ Host 与 relay Viewer 复用同一个纯 presentation reducer 保存远端 `View 首屏就是工具，不做营销页。视觉采用中性深灰工作台，绿色表示 direct、琥珀色表示 relay/警告、红色表示失败，避免单一蓝紫色主题；浏览器 favicon 复用页头 `MonitorUp` 的深绿底、绿色显示器和上行箭头语言。 - 访问入口：未鉴权根路径、`/join` 和合法 code-only `/r/{code}` 显示“站点访问”与“站点口令”；malformed `/r/...` 直接显示房间号必须是 `1000..9999` 四位数字，其他未知路径只显示“无法访问”。有效 `/r/{code}#v={grant}` 直接进入；其余 Viewer 通过 site access 后以 code-only 方式进入。未分配、已回收或已过期的 well-formed code 显示“房间不存在或已过期”，不展开或提供密码输入；现存房间的 `ROOM_ACCESS_DENIED` 保留“输入房间密码”和使用 Host 邀请的动作。已有有效 site-access cookie 不重复显示整站验证；站点口令和房间密码不共享输入值或授权状态。 -- 分享者根页面：顶部显示“{分享者显示名} 的屏幕”和房间状态，并在标题旁显示可单独复制的紧凑四位房间号；Viewer 同一位置显示权威 Host 名称，不再在标题下重复。主区域保持稳定的 16:9 舞台。未发布、已停止或可恢复错误状态时，舞台内以同等尺寸和权重显示“开始分享”和“加入房间”，中间只用“或”与分隔线表达发送/接收；固定 action row 只能包含这两个按钮及分隔，disclosure 是紧随其后的 sibling，不得参与 action-row track sizing。前者直接开始现有分享流程，后者在当前舞台内展开共享的房间码表单，提交后才导航到 `/r/{code}`。按钮同步 `aria-expanded`/`aria-controls`，展开后输入获得焦点，DOM 与焦点顺序保持触发器后接表单；窄屏允许 disclosure 自己换行，不改变两个 action 的矩形或产生横向滚动。不增加首页、向导或拓扑选择。直播时舞台显示本地预览及独立全屏图标，底部保留停止、同步暂停音视频和质量档位。邀请控制使用紧凑 copy-first 布局：包含 22 字符 grant 的完整短 URL 保持可见，复制是主动作；更新和撤销使用有 tooltip/accessible name 的 familiar icons，房间号复制、`公开 | 私密` code-entry segmented control 与可选密码编辑保持职责分离。页面失焦仅暂停本地 video element，并在舞台提示“本地预览已暂停，分享仍在继续”；媒体暂停则禁用 capture stream 轨道，两者都保留现有连接。 +- 分享者根页面：顶部显示“{分享者显示名} 的屏幕”和房间状态，并在标题旁显示可单独复制的紧凑四位房间号；Viewer 同一位置显示权威 Host 名称，不再在标题下重复。主区域保持稳定的 16:9 舞台。未发布、已停止或可恢复错误状态时，舞台内以同等尺寸和权重显示“开始分享”和“加入房间”，中间只用“或”与分隔线表达发送/接收；固定 action row 只能包含这两个按钮及分隔，disclosure 是紧随其后的 sibling，不得参与 action-row track sizing。前者直接开始现有分享流程，后者在当前舞台内展开共享的房间码表单，提交后才导航到 `/r/{code}`。按钮同步 `aria-expanded`/`aria-controls`，展开后输入获得焦点，DOM 与焦点顺序保持触发器后接表单；窄屏允许 disclosure 自己换行，不改变两个 action 的矩形或产生横向滚动。不增加首页、向导或拓扑选择。直播时舞台只显示无媒体 controls 的本地预览，权威停止、同步暂停音视频、切换来源和质量档位保持为明确的页面动作。邀请控制使用紧凑 copy-first 布局：包含 22 字符 grant 的完整短 URL 保持可见，复制是主动作；更新和撤销使用有 tooltip/accessible name 的 familiar icons，房间号复制、`公开 | 私密` code-entry segmented control 与可选密码编辑保持职责分离。页面失焦仅暂停本地 video element，并在舞台提示“本地预览已暂停，分享仍在继续”；媒体暂停则禁用 capture stream 轨道，两者都保留现有连接。 - `/join`：保留为可直接访问的回退入口，并复用同一个四位房间码表单；一个纯 route parser 返回 `host | join | viewer | malformed-room | unknown`，只把精确 `/r/{1000..9999}` 当成 Viewer route。输入保留用户原文，`inputMode="numeric"` 只提示软键盘，提交时由共享 schema 拒绝而不是 `trim`、删除非数字或接受 Unicode 数字。通过 site access 后输入合法 code 并进入 `/r/{code}`，后续行为由 code-only admission 决定。 -- `/r/{code}` 观看者：首次解析合法 fragment grant 后立即清理地址栏并直达；仅有 code 时先通过 site access，再按 `open/private` policy 进入。私密房间统一返回通用拒绝并允许尝试密码，不披露房间是否配置了密码。鉴权后主区域最大化远端视频，标题旁显示可复制的紧凑房间号，顶部保留连接状态，并提供本地音量、静音、全屏和重试控制；Host 主动暂停时复用视频舞台遮罩显示“分享者已暂停”，恢复后立即移除。视频舞台用单一派生 presentation reducer 合成 access、Host presence、route prepare/active、peer/SFU connection、remote track、首个 composited frame、autoplay rejection 和 bounded failure。支持时用 `requestVideoFrameCallback()` 确认第一帧已提交，旧浏览器回退到当前代 decoded-frame delta 与 video readiness；`playing`、track object 或 ICE `connected` 单独都不足以结束占位层。连接阶段占用固定布局且不显示虚假百分比，播放按钮只处理实际 autoplay rejection，失败重试只调用现有有界 controller 入口。 +- `/r/{code}` 观看者：首次解析合法 fragment grant 后立即清理地址栏并直达；仅有 code 时先通过 site access，再按 `open/private` policy 进入。私密房间统一返回通用拒绝并允许尝试密码，不披露房间是否配置了密码。鉴权后主区域最大化远端视频，标题旁显示可复制的紧凑房间号，顶部保留连接状态；同一个 video element 使用浏览器原生 controls 处理本地播放、音量和全屏，页面只保留“重新连接媒体”。Host 主动暂停时复用视频舞台遮罩显示“分享者已暂停”，恢复后立即移除。视频舞台用单一派生 presentation reducer 合成 access、Host presence、route prepare/active、peer/SFU connection、remote track、首个 composited frame、autoplay rejection 和 bounded failure。支持时用 `requestVideoFrameCallback()` 确认第一帧已提交，旧浏览器回退到当前代 decoded-frame delta 与 video readiness；`playing`、track object 或 ICE `connected` 单独都不足以结束占位层。当前代 composited frame 清除同 revision 的旧 connecting/reconnecting 展示；页面恢复先 rebaseline decoded stall。手动重连只重建同一 P2P parent 或同一 SFU subscription，不主动重选线路。 - 首帧观察绑定当前 assignment/connection/media-binding generation 以及适用的 publication generation；`srcObject` 或 route identity 变化立即使旧回调和旧 decoded baseline 失效，迟到回调不能把新路线标成已播放。 - 移动端：视频保持稳定宽高比，控制与状态移到视频下方，任何文字和按钮不得溢出或遮挡。 两端默认显示紧凑在线 roster；Viewer 端复用同一权威 participant snapshot，只列出当前 Viewer（含本机）并在新 snapshot 到达时整体替换，停止分享仅移除已离线 Host，鉴权失效、房间关闭或页面卸载则清空。Host 与 Viewer 的本机显示名在界面中统一标为“昵称”，默认只读，铅笔按钮紧贴昵称并只切换当前控件的输入、保存和取消，唯一名称仅显示名称，重名时才追加 room-scoped `peerId` 短后缀。一个不持久化的“显示连接详情”复选框按页展开媒体方式、传输协议、候选路径、浏览器实际暴露的本地/远端候选地址、codec、RTT、RTP 丢包率、编解码耗时及逐边统计；地址与端口只从当前页面 `RTCPeerConnection.getStats()` 精确关联的 selected pair 读取，浏览器隐藏或缺失任一字段就不显示，且不进入质量证据、信令、服务端、日志或持久化。界面不提供诊断文件下载。真实连接失败、音频缺失、持续画质受限和需要用户动作的告警始终可见。显示名/roster 不进入 access credential 或服务器持久状态。 -用户可见的非直播状态使用稳定、真实的阶段词：分享端使用“已停止分享”；观看端使用“等待开始分享”“正在分配线路”“正在建立 P2P”“正在连接备用线路”“正在接收画面”“需要点击播放”“正在恢复连接”或明确失败。阶段只描述当前用户可感知动作，不暴露房间寿命、媒体代次、parent identity 或内部迁移步骤。`room-closed` 显示“房间已关闭”或“房间已过期”，不得误写成“分享已结束”。服务端 raw `message` 永不直接渲染；单一纯 reducer 按下表优先级把 typed current-generation fact 派生为文案、遮罩、保留画面和动作，较旧 revision/generation 输入直接丢弃： +用户可见的非直播状态使用稳定、真实的阶段词：分享端使用“已停止分享”；观看端使用“等待开始分享”“正在分配线路”“正在建立 P2P”“正在连接 SFU”“正在接收画面”“播放已暂停”“正在恢复连接”或明确失败。阶段只描述当前用户可感知动作，不暴露房间寿命、媒体代次、parent identity 或内部迁移步骤。`room-closed` 显示“房间已关闭”或“房间已过期”，不得误写成“分享已结束”。服务端 raw `message` 永不直接渲染；单一纯 reducer 按下表优先级把 typed current-generation fact 派生为文案、遮罩、保留画面和动作，较旧 revision/generation 输入直接丢弃： | 优先级 | 权威输入 | 展示与动作 | | ---: | --- | --- | @@ -318,7 +318,7 @@ Token-free SFU standby prewarm 保持可选且不创建 participant、publicatio 按钮使用 `lucide-react` 图标并提供可访问名称和 tooltip。动态状态区域固定最小尺寸，避免连接状态变化导致布局跳动。在 1440、1024、780、520、390、375 与 320 CSS px 下，closed/open/validation-error 三种 disclosure 状态的两个首屏 action 都须同排等宽、触控高度至少 44 px且前后 `getBoundingClientRect()` 尺寸不变；输入获得焦点，文字不溢出，详情不遮挡视频且 `scrollWidth <= clientWidth`。 -观看端使用原生 `range` 展示 `0%` 至 `100%`，并把归一化值直接写入当前 `