docs(media): align phase truth (#259)
This commit is contained in:
AAA authored and GitHub committed
1 parent
52dca9a3aa
commit
cbaa689a0f
14 files changed
+773
-1769
No files matched your search
@@ -1,6 +1,6 @@
|
||||
# ADR-0005: Automatic Hybrid Media Routing
|
||||
|
||||
- Status: The route model is accepted and deployed in exact release `1d87615`
|
||||
- Status: The route model is accepted and deployed in exact release `af348ee`
|
||||
on strict v12, including Browser SFU ICE-server isolation, no product
|
||||
diagnostic-download UI, and the retained Host-only acceptance snapshot.
|
||||
- Date: 2026-08-20
|
||||
@@ -271,6 +271,14 @@ second mutable graph.
|
||||
state sampler is separate from the pending candidate's short-lived first-frame
|
||||
observer and does not treat bitrate, FPS, track availability, or SFU layer
|
||||
choice as route authority.
|
||||
- Browser page resume/visibility recovery rebaselines this same deadline before
|
||||
another no-progress sample can invalidate an edge. Frozen JavaScript wall time
|
||||
is not media-failure proof. A newly composited current-generation frame clears
|
||||
stale local recovery presentation but does not create route authority.
|
||||
- Manual media reconnect remains on the current exact route: a peer rebuild uses
|
||||
the same parent, and an SFU reconnect reconstructs the same subscription.
|
||||
Neither action performs quality selection or active reparenting. Only actual
|
||||
recovery exhaustion reports route failure and enters normal reconciliation.
|
||||
- Authoritative pause aborts the pending child operation, including its current
|
||||
candidate and reservations, keeps the active graph, suppresses decoded-frame-
|
||||
stall decisions, and leaves new participants waiting. Resume always wakes a
|
||||
@@ -365,18 +373,14 @@ deployment-wide.
|
||||
|
||||
## Current Source And Deployment Boundary
|
||||
|
||||
Production runs exact deployed application/runtime revision
|
||||
`1d8761528d0dba43fb6d818df3934483ba2f5340`, release `1d87615`; canonical `main`
|
||||
contains newer source pending deployment, including the active-SFU cadence correction
|
||||
and exact Browser SFU ICE-server-isolation implementation
|
||||
`ae09c760adec76fd26da611d4928486d105c6d3b`. Both run strict `screener-v12` and
|
||||
the direct/STUN peer plus LiveKit SFU/UDP route model. Browser SFU PCs use empty
|
||||
external ICE-server lists. Chrome 151 on that exact production verified one
|
||||
publisher and one subscriber advancing frames over LiveKit UDP; the controlled
|
||||
exact-candidate rollback/SFU commit gate is also closed. The operation owner
|
||||
remains only `route`. Exact evidence is owned by the
|
||||
[verification ledger](../verification-status.md); broader heterogeneous-network
|
||||
and SFU lifecycle validation remains open.
|
||||
Production and canonical `main` run exact application/runtime revision
|
||||
`af348ee1d508a3af02b18a7f46c461953798e19d`, release `af348ee`, on strict
|
||||
`screener-v12` with the direct/STUN peer plus LiveKit SFU/UDP route model.
|
||||
Browser SFU PCs use empty external ICE-server lists. The controlled exact-
|
||||
candidate rollback/SFU commit and active-SFU cadence gates are closed. The
|
||||
operation owner remains only `route`; broader heterogeneous-network and SFU
|
||||
lifecycle validation remains open in the
|
||||
[verification ledger](../verification-status.md).
|
||||
|
||||
## Acceptance Boundary
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# ADR-0007: LiveKit-Owned SFU Representation Adaptation
|
||||
# ADR-0007: Framework-Owned Media Quality Adaptation
|
||||
|
||||
- Status: Accepted, implemented, and deployed
|
||||
- Date: 2026-08-19
|
||||
@@ -6,146 +6,101 @@
|
||||
|
||||
## Context
|
||||
|
||||
Direct and peer paths already have independent `RTCPeerConnection` congestion
|
||||
controllers. The SFU path is different: one Host publication serves several
|
||||
subscribers, so a weak subscriber needs a lower encoded representation without
|
||||
lowering every healthy subscriber or creating one encoder per Viewer.
|
||||
Screener sends one realtime game-screen source through independent P2P
|
||||
connections and, when needed, one shared LiveKit publication. WebRTC and
|
||||
LiveKit already own congestion control, encoder adaptation, retransmission,
|
||||
simulcast construction, subscriber bandwidth estimation, and layer selection.
|
||||
|
||||
A single non-scalable full-resolution representation cannot provide that spatial
|
||||
downshift because an SFU forwards encoded packets and does not transcode them.
|
||||
The earlier production A/B compared an always-active two-encoding publication
|
||||
against its lower representation disabled. It showed that the always-active
|
||||
lower representation competed with the full representation on that constrained
|
||||
Host-to-SFU path; it did not test
|
||||
LiveKit-owned demand-driven layer control and does not justify a permanent
|
||||
single-encoding contract.
|
||||
Earlier experiments added a Screener-defined lower SFU encoding and treated
|
||||
quality preference as an application policy. Production evidence showed that
|
||||
an always-active lower encoding can compete with the highest encoding on a
|
||||
constrained Host-to-SFU path. It did not justify replacing LiveKit's native
|
||||
adaptation with a single HIGH representation or another application controller.
|
||||
|
||||
The earlier application-layer controller discussion started from the false
|
||||
premise that LiveKit/WebRTC did not already own per-subscriber bandwidth
|
||||
estimation and layer selection. Built-in media control is the default owner.
|
||||
Screener supplies the bounded representations and product ceilings but does not
|
||||
reimplement that controller. A resource-constrained Host can explicitly choose
|
||||
a lower existing share profile; the product does not protect it by silently
|
||||
removing the representation required by constrained Viewers.
|
||||
The product needs standard game-motion intent and user-selected ceilings, but
|
||||
it does not need a second bitrate, resolution, FPS, or layer-control system.
|
||||
|
||||
## Decision
|
||||
|
||||
1. Direct and peer paths retain independent stock WebRTC congestion control.
|
||||
Viewer feedback is never aggregated into a room-wide target.
|
||||
2. The Browser Host SFU publication provides two bounded VP8 screen-share
|
||||
representations: one half-resolution encoding and the original full-resolution
|
||||
encoding. The lower representation keeps at most 30 fps and derives its
|
||||
bitrate from the pinned SDK's pixel-and-frame-rate formula; the representation
|
||||
count is fixed and does not grow with Viewer count.
|
||||
3. LiveKit Dynacast and its per-subscriber stream allocator/BWE own publication
|
||||
layer activation and the actual layer forwarded to each SFU subscriber.
|
||||
Screener does not implement a quality score, bandwidth estimator, periodic
|
||||
layer controller, or per-Viewer encoder.
|
||||
4. A subscriber's `HIGH` request is a ceiling, not a forced delivery layer. A
|
||||
weak downlink may receive the lower representation while another subscriber
|
||||
receives the highest available representation when the pinned LiveKit stack
|
||||
can sustain that contract.
|
||||
5. AdaptiveStream is a display-demand input, not a network detector. Current
|
||||
Screener subscribers keep it disabled because any Viewer may become a relay
|
||||
and LiveKit cannot see that Viewer's peer children. Network adaptation uses
|
||||
the server stream allocator/BWE instead.
|
||||
6. ADR-0005 remains the only route owner. Layer choice does not change topology,
|
||||
endpoint capacity, SFU admission, or decoded-stall recovery authority.
|
||||
|
||||
Production release `e14eb0e` left the screen-share layer list unset, so pinned
|
||||
client `2.22.0` supplies a same-FPS half-resolution representation beside the
|
||||
original. A live mixed P2P/SFU observation confirmed that the 60 fps lower
|
||||
representation can consume enough Host-to-SFU budget to starve the highest
|
||||
representation. The accepted correction keeps LiveKit's two-layer ownership but
|
||||
supplies one half-resolution lower preset with
|
||||
`lowFps = min(30, highFps)` and the SDK's existing proportional bitrate formula.
|
||||
The two encodings remain RIDs `q,h`; LiveKit's protocol labels them `LOW,MEDIUM`,
|
||||
while a subscriber's default `HIGH` ceiling still selects the highest available
|
||||
encoding. Dynacast, server send-side BWE, and disabled AdaptiveStream remain
|
||||
unchanged. Production release `be53c4d` deploys that correction.
|
||||
|
||||
Current source derives the lower preset from the actual capture dimensions and
|
||||
passes it through LiveKit's public publication options. Retained publication
|
||||
options give LiveKit the same preset on its own source-replacement or republish
|
||||
boundary; Screener does not mutate the active lower encoding behind LiveKit's
|
||||
Dynacast sender lock.
|
||||
|
||||
## Pinned Physical Result
|
||||
|
||||
The 2026-08-25 gate used LiveKit server `1.13.5`, client `2.22.0`, Chrome 151,
|
||||
a continuously changing VP8 source, and one full-resolution plus half-resolution
|
||||
screen-share publication. Manual maximum/lower selection delivered `1280x720`
|
||||
and `640x360` at about 30 fps, and the same connection recovered from the lower
|
||||
to the full-resolution representation.
|
||||
|
||||
With the server default receiver-side BWE, an approximately 0.96 Mbps subscriber
|
||||
remained on HIGH for about 21 seconds and ended at zero decoded fps. Enabling
|
||||
LiveKit `congestion_control.use_send_side_bwe` made the native stream allocator
|
||||
select LOW; by the third steady window it delivered `640x360` at about 29 fps and
|
||||
88 KB/s. A fresh unshaped subscriber immediately received HIGH. The constrained
|
||||
connection's automatic upgrade after removing shaping was not isolated because
|
||||
Chrome bound that test condition when creating the PeerConnection; manual and
|
||||
AdaptiveStream same-connection LOW-to-HIGH recovery were independently proven.
|
||||
|
||||
Dynacast paused layers with no subscribers after about five seconds. Lower-only
|
||||
demand disabled the full-resolution encoding, while maximum demand kept both
|
||||
encodings active; the second VP8 simulcast encode is therefore an accepted
|
||||
bounded cost, not eliminated by Dynacast. AdaptiveStream selected the full
|
||||
representation for a large attached element, the lower representation for a
|
||||
small element, paused a hidden element, and recovered when visible. With no
|
||||
attached element it received only the lower representation, confirming that it
|
||||
cannot own relay ingress. This gate established the framework combination of
|
||||
VP8 screen-share simulcast, Dynacast, server send-side BWE, and
|
||||
`adaptiveStream: false`; the later production observation amends only the lower
|
||||
representation's FPS and proportional bitrate.
|
||||
1. Browser video uses VP8 across direct, browser-relay, and SFU paths. Codec
|
||||
selection, backup codecs, runtime codec switching, and codec state in the
|
||||
room wire are not product features.
|
||||
2. Display video uses the standard `contentHint = "motion"`; display audio uses
|
||||
`contentHint = "music"`. The hint expresses content intent and does not
|
||||
promise a resolution, frame rate, bitrate, encoder, or hardware path.
|
||||
3. The three recommended profiles and advanced controls supply capture and
|
||||
sender ceilings plus `degradationPreference`. Every sender reads back the
|
||||
parameters that the browser actually accepted.
|
||||
4. Direct and peer paths leave media adaptation to each `RTCPeerConnection`.
|
||||
After accepting an answer, the Host reapplies the current video profile to
|
||||
the negotiated sender because the browser may replace or rewrite encoding
|
||||
parameters during negotiation.
|
||||
5. The SFU publisher sets VP8, the selected HIGH ceiling, and degradation
|
||||
preference, but does not set `screenShareSimulcastLayers`, mutate lower
|
||||
encodings, or select a subscriber layer. Pinned LiveKit defaults own the
|
||||
representation set and source-replacement/republish behavior.
|
||||
6. LiveKit Dynacast and server send-side BWE remain enabled. AdaptiveStream
|
||||
remains disabled because a Viewer may relay its received track to peer
|
||||
children; local DOM size or visibility cannot represent that downstream
|
||||
demand.
|
||||
7. Quality measurements are diagnostic. Bitrate, resolution, FPS, RTT, jitter,
|
||||
loss, freeze counters, codec, and limitation reason do not trigger parent
|
||||
selection, relay abdication, periodic rebalancing, or route changes. Only
|
||||
hard connection failure, the existing non-paused decoded-frame stall, parent
|
||||
departure, or capacity invalidation can make an active edge unusable.
|
||||
8. A healthy decoded route remains sticky. Manual media reconnect rebuilds the
|
||||
current exact P2P parent or current SFU subscription; it does not search for
|
||||
a better parent. If current-route recovery genuinely exhausts, ADR-0005's
|
||||
existing controller tries other eligible P2P parents before SFU.
|
||||
|
||||
## Evidence Boundary
|
||||
|
||||
The retained exact-production A/B used Chrome 151 with an always-active ordered
|
||||
`q,h` publication. The 1904x928 `h` stream was about 9.2 fps / 1.37 Mbps with
|
||||
both encodings active and about 23.3 fps / 3.32 Mbps with `q` inactive. Capture
|
||||
remained near 60 fps, encode cost was about 4.46 ms per frame, loss and
|
||||
retransmission were zero, and available outgoing bitrate was about 4.81 Mbps.
|
||||
This rejects always-on `LOW` for that path. It does not establish Dynacast,
|
||||
AdaptiveStream, heterogeneous subscriber behavior, or real-game performance.
|
||||
Chrome 151 production measurements established three durable facts:
|
||||
|
||||
Quality evidence remains diagnostic. Track availability, bitrate, resolution,
|
||||
FPS, RTT, jitter, loss, codec and current LiveKit layer cannot authorize route
|
||||
mutation or predict another parent.
|
||||
- direct and browser-relay VP8 can sustain about 60 fps after stock bandwidth
|
||||
estimation warms up; sender ceilings are not startup guarantees;
|
||||
- forcing an always-active lower SFU encoding can reduce the highest encoding
|
||||
on the same Host-to-SFU congestion budget; and
|
||||
- pinned LiveKit `2.22.0` plus server `1.13.5` can select lower and higher VP8
|
||||
screen-share representations per subscriber when server send-side BWE is
|
||||
enabled.
|
||||
|
||||
Separate content-hint probes showed that `motion` changes Chromium's adaptation
|
||||
tradeoff and may spatially downscale to preserve motion. That is the standard
|
||||
behavior requested for games, not a quality floor. Real-game readability,
|
||||
weaker Hosts, heterogeneous devices, and public-network SFU quality still
|
||||
require physical evidence.
|
||||
|
||||
## Consequences
|
||||
|
||||
- A weak SFU Viewer can receive a real lower spatial representation when the
|
||||
framework selects it; the SFU does not transcode.
|
||||
- The Host publishes at most two SFU representations regardless of Viewer
|
||||
count. Dynacast pauses representations above aggregate demand and all
|
||||
representations when no subscriber remains; maximum demand keeps both VP8
|
||||
simulcast encodings active.
|
||||
- Host encode capacity is handled through the existing explicit share profiles,
|
||||
not by falling back to one encoding and abandoning constrained subscribers.
|
||||
- An SFU-fed relay keeps a subscription ceiling sufficient for its subtree;
|
||||
local DOM visibility alone cannot pause that ingress.
|
||||
- A configured 60 fps or bitrate remains a ceiling, not a delivery guarantee.
|
||||
- Screener owns fewer media mechanisms and follows the pinned frameworks'
|
||||
supported control surfaces.
|
||||
- A constrained Viewer may receive a lower LiveKit representation without
|
||||
lowering every subscriber, subject to the publisher and network actually
|
||||
sustaining the framework contract.
|
||||
- A Host may need to lower its explicit share profile when encoder or uplink
|
||||
capacity is insufficient; Screener does not silently remove constrained
|
||||
Viewer support.
|
||||
- Quality-based topology optimization remains unimplemented. Accepting it later
|
||||
requires evidence for alternative-path measurement and a separate route-model
|
||||
decision.
|
||||
|
||||
## Stop Lines
|
||||
|
||||
- Do not create one encoder or representation per Viewer.
|
||||
- Do not add application quality scoring, periodic polling, manual layer
|
||||
switching, or quality-driven reparenting.
|
||||
- Do not use AdaptiveStream without the LiveKit track attachment contract.
|
||||
- Do not infer Dynacast success from configuration or source inspection alone.
|
||||
- Do not apply SFU layer policy to ordinary direct or peer paths.
|
||||
- No Screener resolution/FPS/bitrate ladder or representation formula.
|
||||
- No manual SFU layer selector, forced single HIGH publication, or per-Viewer
|
||||
encoder.
|
||||
- No quality score, all-pairs probing, periodic rebalancing, or speculative
|
||||
parent switching.
|
||||
- No AdaptiveStream while a subscriber can relay the track.
|
||||
- No quality or hardware claim from configured options alone.
|
||||
|
||||
## References
|
||||
|
||||
Primary sources checked through 2026-08-25:
|
||||
|
||||
- [MediaStreamTrack Content Hints](https://www.w3.org/TR/mst-content-hint/)
|
||||
- [WebRTC](https://www.w3.org/TR/webrtc/)
|
||||
- [WebRTC Statistics](https://www.w3.org/TR/webrtc-stats/)
|
||||
- [LiveKit video simulcast and Dynacast](https://docs.livekit.io/transport/media/advanced/)
|
||||
- [LiveKit selective subscription](https://docs.livekit.io/transport/media/subscribe/)
|
||||
- [LiveKit client 2.22.0 room options](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/options.ts)
|
||||
- [LiveKit client 2.22.0 AdaptiveStream track attachment](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/track/RemoteVideoTrack.ts)
|
||||
- [LiveKit client 2.22.0 subscriber quality ceiling](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/track/RemoteTrackPublication.ts)
|
||||
- [LiveKit client 2.22.0 Dynacast handling](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/participant/LocalParticipant.ts)
|
||||
- [LiveKit server 1.13.5 per-subscriber layer application](https://github.com/livekit/livekit/blob/v1.13.5/pkg/rtc/subscribedtrack.go)
|
||||
- [LiveKit server 1.13.5 Dynacast quality aggregation](https://github.com/livekit/livekit/blob/v1.13.5/pkg/rtc/dynacast/dynacastqualityvideo.go)
|
||||
- [LiveKit server 1.13.5 enabled-quality generation](https://github.com/livekit/livekit/blob/v1.13.5/pkg/rtc/dynacast/dynacastmanagervideo.go)
|
||||
- [LiveKit client 2.22.0 encoding construction](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/participant/publishUtils.ts)
|
||||
- [LiveKit server 1.13.5 layer forwarding](https://github.com/livekit/livekit/blob/v1.13.5/pkg/sfu/forwarder.go)
|
||||
+14
-15
@@ -3,7 +3,7 @@
|
||||
Last verified against upstream documentation: 2026-08-24.
|
||||
|
||||
This page records production running exact deployed application/runtime revision
|
||||
`be53c4d5794d38d5d406c876309958d7117ee601`, release `be53c4d`, and the single
|
||||
`af348ee1d508a3af02b18a7f46c461953798e19d`, release `af348ee`, and the single
|
||||
Browser `screener-v12` contract. Product direction and pending work are owned by
|
||||
[project memory](./project-memory.md) and [the TODO ledger](./todo.md).
|
||||
|
||||
@@ -13,9 +13,8 @@ Caddy or nginx; ordinary peer ICE advertises only STUN, while Browser LiveKit
|
||||
PCs configure no external ICE server and retain LiveKit-signaled UDP candidates.
|
||||
LiveKit supplies bounded SFU capacity. Normal media remains distributed
|
||||
through direct or peer edges whenever those paths work. Production includes
|
||||
the Browser SFU ICE isolation and two-representation model. Chrome 151 physically
|
||||
verified UDP publication/subscription and native full/lower selection; the
|
||||
deployed lower-representation cost correction still needs mixed-route media evidence.
|
||||
Browser SFU ICE isolation and leaves representation construction and selection
|
||||
to pinned LiveKit with Dynacast and server send-side BWE.
|
||||
|
||||
A deployment may additionally provide one dedicated single-node LiveKit process
|
||||
as the current controller's automatic final media fallback. This capacity is
|
||||
@@ -26,21 +25,21 @@ ICE/UDP only. Ordinary peer ICE remains STUN-only and production coturn uses the
|
||||
tracked STUN-only configuration with TCP/TLS disabled. The source and production
|
||||
configure no TURN, ICE/TCP, media TCP, or TLS-relayed media.
|
||||
|
||||
Production runs exact `be53c4d5794d38d5d406c876309958d7117ee601`, release
|
||||
`be53c4d`, from `/opt/screener/releases/be53c4d`. The immutable runtime tar
|
||||
Production runs exact `af348ee1d508a3af02b18a7f46c461953798e19d`, release
|
||||
`af348ee`, from `/opt/screener/releases/af348ee`. The immutable runtime tar
|
||||
SHA-256 is
|
||||
`5c174c30ffacd98738a364b825b915b84432823c261272684937361be2451144`.
|
||||
`4922cc31d7caaad7c5412b1f2fe1d73f788d16d92386efad6eac358a97b8a78e`.
|
||||
Its 39-file path/size/hash manifest SHA-256 is
|
||||
`c34dba43a25c6b8447e1153a5be3191bf82cff94117ed31edc325d3f7cf8541f`.
|
||||
`5b397809b664b9a55885ac603df615b537bf51276e7274416820870956242d7d`.
|
||||
Local and public `/healthz` return 200; Screener, LiveKit, coturn, and nginx are
|
||||
active with `NRestarts=0`. The public main Browser asset is
|
||||
`assets/index-BKEVI7Ux.js` with SHA-256
|
||||
`22114fc8a26b3f03f37973555433d1e73d52d1c7b76fa3c7cc597500823bd62d`.
|
||||
`assets/index-BvuZY6Hc.js` with SHA-256
|
||||
`e8fc0500496ee101031a9cfebfcc22d3b69e66d2cac5e875448b56620dbb21c5`.
|
||||
|
||||
The release deploys the single Browser `screener-v12` wire, fixed VP8 for Browser
|
||||
direct, browser-relay, and SFU video, one original plus one half-actual-resolution
|
||||
SFU encoding capped at 30 fps with proportional bitrate, Dynacast and server
|
||||
send-side BWE without an application layer selector, no video `contentHint`, no
|
||||
direct, browser-relay, and SFU video, pinned LiveKit default screen-share
|
||||
representations, Dynacast and server send-side BWE without an application layer
|
||||
selector, video `contentHint = "motion"`, no
|
||||
codec UI/state/wire, random four-digit memory rooms with a 24-hour dormant lease,
|
||||
a room-lived 22-character grant,
|
||||
orthogonal `open | private` grant/code admission,
|
||||
@@ -284,7 +283,7 @@ lifetime. An actively connected Host prevents expiry; explicit stop or Host
|
||||
disconnect starts the lease, and only the exact Host token renews it before
|
||||
expiry. Viewer activity never renews ownership. `ROOM_DATABASE_PATH` and
|
||||
`ROOM_TTL_SECONDS` fail startup even when blank.
|
||||
Production `be53c4d` accepts 1 through 20 and explicitly selects 20. This is an
|
||||
Production `af348ee` accepts 1 through 20 and explicitly selects 20. This is an
|
||||
admission limit, not evidence that every publisher, network, or quality profile
|
||||
can sustain that many streams.
|
||||
`ENDPOINT_MEDIA_COPY_CAPACITY` defaults to 2 and accepts only 1, 2, or 3. It is
|
||||
@@ -297,7 +296,7 @@ must fail or wait before a fourth endpoint copy is issued.
|
||||
Supplying the removed `MAX_PEER_RELAY_DOWNSTREAM_EDGES`, even blank, fails
|
||||
startup.
|
||||
|
||||
Production release `be53c4d` runs the deployed server and Browser assets
|
||||
Production release `af348ee` runs the deployed server and Browser assets
|
||||
atomically on `screener-v12`; every stale Browser or executable-sender wire fails
|
||||
before room authority. Native senders and helpers are outside this release.
|
||||
|
||||
|
||||
+114
-43
@@ -4,56 +4,127 @@ Last updated: 2026-08-25
|
||||
|
||||
## Current Product Truth
|
||||
|
||||
- Build private, low-latency game screen sharing for one broadcaster and a small group of trusted friends. Public or large broadcasts belong on OBS/Twitch-class services.
|
||||
- A room supports one Host and up to `20` authenticated Viewers. Admission is independent of endpoint fanout and SFU capacity; the event-driven route controller must remain efficient at that bound without all-pairs probing.
|
||||
- Authenticated Hosts and shipped clients are trusted media participants in these private rooms; route authority, generation fencing, and server admission remain fail-closed.
|
||||
- Rooms use one bounded, process-memory model: a random free code from `1000..9999`, default 24-hour configurable dormant lease, and no database or cross-restart recovery. Active sharing never expires; stop/disconnect starts the lease, the exact Host token may resume before expiry, Viewer activity does not renew it, and expiry releases the code. Process restart invalidates every room, token, invitation, password verifier, and participant.
|
||||
- Viewer grants and code entry are orthogonal. Each room has one random 128-bit/22-character grant digest bound to that exact in-memory incarnation; it has no independent expiry and ends on reclamation, restart, rotate, or revoke. Every new room creates that invitation and defaults code entry to public `open` with no password; the only other code-entry state is `private`. A private room with no password is invitation-only, while setting a password also permits matching code-only entry. An unallocated/reclaimed/expired code returns `ROOM_NOT_FOUND` without a password prompt, while other existing-room denials remain `ROOM_ACCESS_DENIED`. The Host browser retains its creation preferences and raw ownership token; exact-token invitation/policy/password management remains available while dormant without starting media or renewing the lease. The server keeps only current-process digests/verifiers and uses no fingerprint. Recreation reapplies preferences but creates a new code and credentials.
|
||||
- Web is the current delivery target. The route release covers Browser Host, Viewer, and relay endpoints; packaged/native senders, capture helpers, shared-encode executables, and their acceptance runs are lowest-priority later work outside this milestone.
|
||||
- Keep routing automatic and media distributed. Direct/peer UDP is preferred; the dedicated LiveKit SFU/UDP path is the only application fallback and is not the default topology. Screener configures no TURN, ICE/TCP, media TCP, or TLS-relayed media path.
|
||||
- Every non-server endpoint uses one server-authoritative steady outbound media-copy capacity: default `2`, statically configurable as `1`, `2`, or `3`. A peer child or the Host publication consumes one slot; upstream receive is free, and SFU subscriber egress is accounted at the server. Browser role/UA/visibility does not create a separate tier, and clients cannot raise the deployment value.
|
||||
- The stable routing invariants are one authoritative upstream per Viewer, an acyclic source-reachable active graph, exact authenticated route attempts, endpoint and server admission, one room-serial child operation, and bounded success, wait, or failure.
|
||||
- One room controller reconciles allocation, child reparenting, and relay drain through the same operation. It finds one waiting Viewer or invalid edge and builds one deterministic tuple list. The operation owns its cursor, current candidate and reservations, fact version, plus one total deadline; only its current candidate may produce media. A prepare names that exact child, selected transport, and server-issued candidate connection ID. A usable old edge remains until the exact candidate child decodes its first new video frame. `settleReady` synchronously commits admission before graph promotion; success commits prepare revision `P`, while failure releases resources, broadcasts rollback `R > P` from one monotonic allocator, and advances the cursor without resetting the deadline. Exhaustion retires hard-invalid, departed-parent, or overflow edges into explicit waiting but retains a healthy bootstrap edge. Bounded-gap preflights server resources, replans the untried suffix after retirement, and appends one ordinary restore candidate when it cut healthy media. A relay with a bad ingress reparents itself as a child and retains its subtree. A disconnected endpoint, or one whose effective downstream capacity falls below its current child count, accepts no excess children; its direct or overflow children enter the same operation and a disconnected endpoint is removed when childless. Authoritative pause aborts the operation, suppresses decoded-progress stall decisions, and resumes from a fresh reconciliation.
|
||||
- One Host publication may serve all SFU subscriptions. Its exact generation owns one ingress handle and exact Viewer subscription handles own egress; later SFU children reuse the publication, while reserved, committed, and draining handles remain charged until the managed room is deleted and proven absent. Actual SFU usage release wakes only rooms that previously lost a candidate to global admission, and SFU-fed and peer-fed endpoints use the same provisional-child transaction. Parent choice hard-filters authority, reachability, cycles, effective capacity, and producer transition, then orders by shallowest result, remaining steady sender capacity, stable join order, and peer identity; server admission is checked at the current cursor. Acyclicity and room admission bound maximum depth; depth is observed but has no separate hard cap. Healthy committed edges remain sticky: the controller repairs waiting, invalid, departed-parent, and overflow children but does not periodically rebalance the graph. Raw IP, claimed NAT type, UA, geography, all-pairs probes, weighted scores, and periodic room rebalancing are not route inputs.
|
||||
- Ordinary peer ICE is STUN-only. Browser LiveKit publisher and subscriber PCs use no external ICE server, retain LiveKit-signaled UDP candidates, and let standard ICE nominate a non-relay pair; deployment STUN remains available to ordinary peers and LiveKit server-side public-IP discovery. This Browser SFU ICE-server isolation avoids the reproduced external-STUN/SFU association conflict without constraining candidate type or count and without adding TURN, another route, a port, or a timeout. Credentials, room secrets, candidate details, and diagnostic data remain private and narrowly scoped. A future strict-firewall transport requires separate evidence and must remain inside LiveKit rather than becoming another Screener route candidate.
|
||||
- Browser peers use existing periodic WebRTC/LiveKit sampling to maintain one exact-route decoded-progress timestamp. New frames or route/connection changes reset it; a non-paused deadline reports once. An active SFU cadence treats an absent track/report or stats failure as no progress without fabricating metrics and stops only on deactivation/teardown. Other current-path stats remain diagnostic and cannot prove another parent better.
|
||||
- ADR-0007's two VP8 SFU representations, Dynacast, server send-side BWE, and disabled AdaptiveStream for potential relays are deployed. The lower representation is half the actual capture dimensions, capped at 30 fps, and uses proportionally derived bitrate. Screener adds no media-layer selector.
|
||||
- Browser capture requests available share audio by default on initial capture and source switch, with an explicit video-only warning when no audio track is returned. Its screen-audio control offers live-switchable 64/128/256 kbps sender ceilings and defaults to 128. The last-wins quality wire owns desired state; each endpoint serially applies and reads back current Host/relay/SFU audio senders, exposes local failures, and gives future senders the latest desired value. Opus remains fixed, and without an applied-ack wire the Host does not claim room-wide atomic convergence. These are configuration ceilings, not fidelity claims.
|
||||
- Browser display video leaves `contentHint` unset so Chromium retains its screen-capture classification; returned audio tracks keep `contentHint = "music"`. Browser video is fixed to VP8 across direct, browser-relay, and SFU paths; codec selection, fallback to another media codec, quality-state/wire fields, and share-lifetime codec branches are not product features. Actual codec/profile/encoder stats remain diagnostic only.
|
||||
- Recommended quality remains exactly `1080p60`, `1080p30`, and `720p30`, with the middle `1080p30` as the accepted default. `480p` means only an advanced `854x480` resolution choice whose FPS and bitrate remain independently selected; it is not a fourth recommended profile.
|
||||
- Two distribution artifacts are accepted only after the current feature, NAT, and real-network ledger is complete. A public-server one-click deployment package installs the current application plus STUN/SFU and proxy components on a user-owned server. A Windows/macOS/Linux local package runs the Host, application server, and local state on the broadcaster's machine without source, Node, or any external Screener/network service; it keeps workable direct P2P but reports the reachability loss imposed by public ingress, TLS, gateway, NAT, or firewall limits rather than promising universal connectivity.
|
||||
- Current UI, presence, route labels, and diagnostics describe observed state only; they do not create route authority.
|
||||
- Before an explicit public-release decision, the private deployment and repository keep only the current product contract. Incompatible wire, configuration, parser, and API changes ship atomically; stale clients fail before authority, replaced compatibility surfaces are deleted, and Git history owns rollback.
|
||||
- A routine application-only deployment verifies one new immutable artifact and switches to it atomically while reusing unchanged environment, service, LiveKit, coturn, reverse-proxy, and firewall configuration. The pre-cutover application release is guaranteed only through bounded health and postflight checks; it has no retention contract afterward and is not a maintained backup. A future change to infrastructure, secrets, persistent data, or irreversible state must define recovery only for the surfaces it actually changes.
|
||||
- Viewer presentation stays on stage until current-generation frame proof, distinguishes typed access/Host/route/media/autoplay/recovery phases, and reserves `Play` for autoplay rejection. Pages expose local connection details but no diagnostic download; the bounded Host-only route snapshot remains for acceptance tooling. Route/quality facts are diagnostic only. Browser ICE owns direct reachability; Screener does not predict ports, classify NATs, probe TCP, synthesize candidates, or reparent healthy edges speculatively.
|
||||
- Screener is private, low-latency game screen sharing for one Host and up to
|
||||
`20` authenticated Viewers, not a public broadcast service.
|
||||
- Web Host, Viewer, and Browser relay are the current delivery surface. Native
|
||||
capture, shared encoding, and distributable server/local packages remain
|
||||
later work.
|
||||
- Rooms are bounded process memory: random free `1000..9999` code, default
|
||||
24-hour dormant lease, active sharing never expires, exact Host token resume,
|
||||
Viewer activity does not renew, and process restart loses every room and
|
||||
credential. There is no database or migration path.
|
||||
- Every room has one 128-bit/22-character Viewer grant bound to that exact room
|
||||
incarnation and independent `open | private` code entry. Private without a
|
||||
password is invitation-only; adding a password also permits matching code
|
||||
entry. Reclamation, restart, rotate, or revoke ends the grant. Missing codes
|
||||
return `ROOM_NOT_FOUND` without a password prompt.
|
||||
- Host creation preferences and raw ownership token stay local to the Host
|
||||
browser. Server authority stores only current-process digests/verifiers and
|
||||
never uses IP, UA, device, or browser fingerprint as identity.
|
||||
|
||||
## Media And Routing
|
||||
|
||||
- Media stays automatic and distributed. Ordinary peer ICE is STUN-only and
|
||||
direct/peer UDP is preferred. The only application fallback is the dedicated
|
||||
LiveKit SFU over UDP; Screener configures no TURN, ICE/TCP, media TCP, or
|
||||
TLS-relayed media.
|
||||
- Browser LiveKit publisher/subscriber PCs use no external ICE server and retain
|
||||
LiveKit-signaled UDP candidates. Deployment STUN remains available to ordinary
|
||||
peers and LiveKit server-side public-IP discovery.
|
||||
- Every non-server endpoint uses one server-authoritative steady outbound copy
|
||||
cap: default `2`, configurable only as `1`, `2`, or `3`. A peer child or the
|
||||
Host publication consumes one slot; upstream receive is free; SFU subscriber
|
||||
egress uses separate server admission.
|
||||
- One room controller owns one committed acyclic source-reachable graph, one
|
||||
event-driven reconciliation loop, and at most one room-serial child operation.
|
||||
The operation owns one deterministic candidate list/cursor, one current
|
||||
candidate and reservations, one fact version, and one total direct-then-SFU
|
||||
deadline.
|
||||
- Candidate filtering enforces current authority, reachability, acyclicity,
|
||||
capacity, transition slots, and server admission. Eligible P2P parents are
|
||||
ordered by resulting depth, remaining capacity, stable join order, and peer
|
||||
identity before SFU.
|
||||
- A candidate commits only after the exact child decodes its first new video
|
||||
frame. Failure releases resources, broadcasts a strictly newer rollback
|
||||
revision, advances the cursor without resetting the deadline, and preserves
|
||||
unaffected healthy branches. A relay with bad ingress reparents itself while
|
||||
retaining its subtree.
|
||||
- Healthy decoded routes remain sticky. There is no periodic rebalancing,
|
||||
quality score, all-pairs probing, parent blacklist, NAT classification, port
|
||||
prediction, or independent depth cap. Quality evidence cannot prove an
|
||||
unconnected parent would be better.
|
||||
- P2P and SFU first use their framework reconnect behavior. P2P recovery tries
|
||||
the same parent, and SFU recovery rebuilds the same publication/subscription.
|
||||
Manual media reconnect also stays on the current exact route. Only actual
|
||||
recovery exhaustion, hard failure, non-paused decoded-frame stall, parent
|
||||
departure, or capacity invalidation enters route reassignment.
|
||||
- Active-path WebRTC/LiveKit sampling owns one exact-route decoded-progress
|
||||
deadline. Route/connection change or new progress resets it; authoritative
|
||||
pause suppresses it. Page resume rebaselines frozen wall-clock time before
|
||||
another no-progress sample can fail a route.
|
||||
|
||||
## Media Quality
|
||||
|
||||
- Browser video is VP8 only. Display video uses `contentHint = "motion"` and
|
||||
display audio uses `contentHint = "music"`. Codec UI/state/wire, backup media
|
||||
codecs, and runtime codec switching do not exist.
|
||||
- Recommended profiles remain exactly `1080p60`, `1080p30`, and `720p30`, with
|
||||
`1080p30` default. `480p` is only an advanced `854x480` resolution choice;
|
||||
advanced FPS and bitrate remain independent.
|
||||
- WebRTC owns direct/peer media adaptation. The Host reapplies the selected video
|
||||
profile after answer negotiation; no periodic application controller exists.
|
||||
- The SFU publisher sets VP8 and the selected ceiling but no custom simulcast
|
||||
layers. Pinned LiveKit defaults own representations, Dynacast owns aggregate
|
||||
demand, and server send-side BWE owns subscriber forwarding. AdaptiveStream
|
||||
stays disabled because any Viewer may relay its received track.
|
||||
- Screen audio requests capture by default and offers live 64/128/256 kbps
|
||||
sender ceilings with 128 default. Peer answers request Opus stereo with a
|
||||
256 kbps receive maximum. SFU publication uses stereo, DTX off, and RED off;
|
||||
disabling RED accepts reduced burst-loss resilience in exchange for bounded
|
||||
publisher traffic.
|
||||
- Configured resolution, FPS, bitrate, preference, codec, and audio ceiling are
|
||||
requests or ceilings. Sender readback and RTCStats are the observable truth.
|
||||
|
||||
## Presentation And Lifecycle
|
||||
|
||||
- Viewer presentation is revision/generation-fenced and keeps the stage stable
|
||||
until a current-generation composited frame. A new current frame clears stale
|
||||
media `connecting/reconnecting` presentation; old callbacks cannot prove a
|
||||
new route.
|
||||
- Viewer local play/pause, volume, mute, and fullscreen are owned by native video
|
||||
controls on one persistent media element. Host preview is a muted, control-free
|
||||
view of the capture stream; only explicit Host actions pause or stop sharing.
|
||||
- Hiding or unfocusing the Host page pauses only the local preview element.
|
||||
Browser/OS capture and background behavior remain platform capabilities; no
|
||||
fake keepalive, silent media, Wake Lock, or timer loop is a product mechanism.
|
||||
|
||||
## Current Source And Production
|
||||
|
||||
- Production runs exact application/runtime revision `be53c4d5794d38d5d406c876309958d7117ee601`, release `be53c4d`, on strict `screener-v12`; canonical `main` contains that deployed application tree plus current truth.
|
||||
- Current source and production admit up to `20` Viewers and implement the uniform `1/2/3` endpoint cap, exact prepare candidate, first-decoded-frame commit, strict rollback revision, decoded-progress stall, one direct/SFU operation deadline, and one event-driven controller. Active-SFU sampling remains live through an absent track/report or stats failure. Ordinary peer ICE is STUN-only and the sole application fallback is SFU/UDP. Exact SFU admission and empty Browser SFU ICE-server options remain unchanged.
|
||||
- Source and production use random four-digit memory rooms, dormant leases, exact-Host resume, restart loss, local preference replay, orthogonal grants/code entry, and no SQLite runtime. V12 uses a 22-character room-incarnation grant, `ROOM_NOT_FOUND`, no diagnostic-file export, and a compact UI showing the complete shorter URL.
|
||||
- Exact production artifacts, services, ports, and any change-scoped infrastructure recovery boundary are owned by [deployment](./deployment.md) and indexed by [status](./status.md).
|
||||
- Source and production require v12, fixed VP8, and no video hint, and derive a half-actual-resolution SFU lower preset capped at 30 fps with proportional bitrate. Both retain Dynacast, server send-side BWE, and disabled AdaptiveStream. Other current media and UI behavior matches the truths above.
|
||||
- Browser SFU subscribers reconcile eligible Host screen publications across connect/activate event ordering, participant/track arrival, and reconnect. Production LiveKit logs only warn/error, and the application service has a bounded same-host LiveKit readiness preflight.
|
||||
- The deployed application tree passed TypeScript, 613 Web tests, both builds, hygiene, two independent reviews, immutable artifact verification, and production postflight. Earlier exact-production gates closed direct/browser-relay VP8 cadence, active-audio ceilings, a single-representation SFU baseline, and one controlled exact-candidate SFU commit; the pinned-stack gate proved native representation selection. The lighter representation's mixed-route quality effect, controlled recovery on this release, representative networks, all-UDP-blocked failure, public SFU 60 fps, and real-game synchronization remain unproven. Exact metrics live in [verification status](./verification-status.md).
|
||||
- Chrome loopback, cap2/cap3, short resource, recovery, MBB, and A/V fixtures are bounded evidence only. They do not establish quality, resource limits, or product policy.
|
||||
- Canonical `main` and production run exact application/runtime revision
|
||||
`af348ee1d508a3af02b18a7f46c461953798e19d`, release `af348ee`, on strict
|
||||
`screener-v12`.
|
||||
- Production uses random memory rooms, 20-Viewer admission, endpoint cap `2`,
|
||||
dedicated LiveKit admission `1` ingress / `20` egress, STUN UDP 3478, LiveKit
|
||||
media UDP 7882, and Web TCP 80/443. Node 8787 and LiveKit 7880 remain private.
|
||||
- Exact artifact and service state are owned by [deployment](./deployment.md)
|
||||
and indexed by [status](./status.md). Physical evidence boundaries are owned
|
||||
by [verification status](./verification-status.md).
|
||||
|
||||
## Current Priority
|
||||
|
||||
1. Verify the deployed lower-representation quality effect on a mixed P2P/SFU room, then repeat controlled active-SFU recovery and finish real-network route, screen-audio, and real-game A/V acceptance. VP8 acceleration, mobile lifecycle, Native, distribution, and repository simplification remain later.
|
||||
1. Integrate and deploy the accepted Viewer lifecycle/native-control/current-
|
||||
route reconnect candidate, then finish representative real-network route,
|
||||
screen-audio, real-game A/V, and mobile lifecycle acceptance.
|
||||
|
||||
The desktop Host background/minimized report is deferred until it reproduces on current production with a real game and synchronized media plus CPU/GPU evidence. The bounded current-Chrome screening found no immediate Host-page lifecycle drop, and removal of the old harmful `contentHint = "motion"` remains a plausible confound rather than a proven background fix. Mobile Viewer playback and assigned-relay survival remain a separate lifecycle gate.
|
||||
Quality-driven parent selection, VP8 hardware evidence, Native sender work,
|
||||
distribution packages, whole-product UI/bilingual polish, and repository-wide
|
||||
simplification remain later decisions in [the TODO ledger](./todo.md).
|
||||
|
||||
## Working Rules
|
||||
|
||||
- Generic task authority, scope, smallest-correct implementation, and current-result delivery are owned by the repo-tracked [`stop-that-shit` skill](../.agents/skills/stop-that-shit/SKILL.md); repository-specific truth and execution rules remain in `AGENTS.md` and `CONTRIBUTING.md`.
|
||||
- Old branches may contribute scoped code or evidence only after reconciliation. Their memory, requirements, ADRs, status, and deployment snapshots never overwrite newer mainline truth.
|
||||
- Preserve dirty, unique, open-stack, and evidence worktrees until their disposition is explicitly decided. Workspace counts are sampled on demand rather than stored here.
|
||||
|
||||
## Source Map
|
||||
|
||||
- Current execution: [status](./status.md)
|
||||
- Executable and held work: [TODO ledger](./todo.md)
|
||||
- Requirements and design: [requirements](./%E9%9C%80%E6%B1%82%E7%90%86%E8%A7%A3.md), [design](./%E6%96%B9%E6%A1%88%E8%AE%BE%E8%AE%A1.md), and [ADRs](./adr/)
|
||||
- Evidence and limits: [research](./research/) and [verification status](./verification-status.md)
|
||||
- Operations: [deployment](./deployment.md), [maintenance](./maintenance.md), and [contributing](../CONTRIBUTING.md)
|
||||
- Generic authority and scope follow the repo-tracked
|
||||
[`stop-that-shit` skill](../.agents/skills/stop-that-shit/SKILL.md).
|
||||
- Canonical `main` and the current truth owners override old branches,
|
||||
worktrees, handoff text, and chat summaries.
|
||||
- Preserve unique or dirty candidates until integration and reference/reparse
|
||||
audits prove cleanup is safe.
|
||||
@@ -3,9 +3,8 @@
|
||||
- Research date: 2026-08-24
|
||||
- Scope: desktop Chrome/Edge Web Host capture while the Host page is
|
||||
unfocused, occluded, backgrounded, or minimized
|
||||
- Status: deferred until a current-production real-game reproduction exists;
|
||||
one bounded current-browser screening did not observe an immediate Host-page
|
||||
background drop, and no new fix is accepted, implemented, or deployed
|
||||
- Status: Host capture report remains unconfirmed; Viewer resume/state lifecycle
|
||||
correction is accepted pending integration and does not claim capture keepalive
|
||||
|
||||
## Current Conclusion
|
||||
|
||||
@@ -53,34 +52,29 @@ configuration. Encoder implementation is a diagnostic variable only after a
|
||||
stable baseline exists; neither codec nor encoder choice is a page-keepalive
|
||||
mechanism.
|
||||
|
||||
The 2026-08-19 report came from exact release `769de201f7cc`, which still set
|
||||
video `contentHint = "motion"`. Current source and production leave the video
|
||||
hint unset; a separate controlled VP8 probe changed from 14.93 fps at 428x208
|
||||
with `motion` to 29.73 fps at 1904x928 without it. That known old-source defect
|
||||
is fixed in the current contract, but it does not prove that the state-dependent
|
||||
background report had the same cause. The physical baseline must therefore
|
||||
start from current production rather than carrying the old report forward as a
|
||||
current regression.
|
||||
The accepted Browser path uses VP8 and `contentHint = "motion"`. A controlled
|
||||
probe showed that this standard game-motion hint can spatially downscale to
|
||||
preserve cadence; that quality tradeoff is independent from page lifecycle and
|
||||
does not explain a state-dependent background report. The physical baseline
|
||||
must start from current production rather than carrying an old report forward
|
||||
as a current regression.
|
||||
|
||||
This investigation is deferred. It reopens only if the report
|
||||
reproduces on current production with a continuously changing real game and
|
||||
synchronized capture, send, receive, decode, CPU, and GPU evidence. Removing the
|
||||
old harmful `motion` hint may have removed a confounding quality defect, but it
|
||||
is not established as a background-lifecycle fix. Mobile Viewer background
|
||||
playback and assigned-relay survival remain a separate browser/OS lifecycle
|
||||
gate and do not inherit this desktop Host result.
|
||||
The Host investigation reopens only if the report reproduces on current
|
||||
production with a continuously changing real game and synchronized capture,
|
||||
send, receive, decode, CPU, and GPU evidence. Mobile Viewer background playback
|
||||
and assigned-relay survival remain a separate browser/OS lifecycle gate and do
|
||||
not inherit this desktop Host result.
|
||||
|
||||
## Current Source Boundary
|
||||
|
||||
The no-video-hint review baseline is
|
||||
`a26eb39dc677003110787b0ed1581c208f894fd7`. Current source and production use
|
||||
fixed VP8, no video hint, and no codec UI, quality state, or wire field; their
|
||||
exact identity is owned by [status](../status.md).
|
||||
Current source and production use fixed VP8, video `contentHint = "motion"`,
|
||||
audio `contentHint = "music"`, and no codec UI, quality state, or wire field;
|
||||
their exact identity is owned by [status](../status.md).
|
||||
The current path does not claim or implement a page-keepalive mechanism, and
|
||||
deployment health supplies no physical background-capture evidence.
|
||||
|
||||
- Current `src/client/media/quality.ts` obtains one `getDisplayMedia()` stream,
|
||||
applies ideal/max capture constraints, leaves the video hint unset, and
|
||||
applies ideal/max capture constraints, sets standard game-motion intent, and
|
||||
applies sender bitrate, frame-rate, and degradation ceilings. Explicit
|
||||
sharing pause and authoritative reconnect re-pause change capture tracks'
|
||||
`enabled` state; neither manufactures foreground activity. Audio
|
||||
@@ -102,6 +96,10 @@ deployment health supplies no physical background-capture evidence.
|
||||
- The short-lived pending-candidate decoded-frame observer reads cumulative RTP
|
||||
progress only for the exact pending route. It neither drives capture nor proves
|
||||
document activity, and it stops when that operation settles.
|
||||
- Viewer resume/visibility recovery rebaselines the existing decoded-stall wall
|
||||
clock and rearms current-frame presentation. It prevents frozen JavaScript
|
||||
time from becoming an immediate route failure; it does not keep a page,
|
||||
capture source, encoder, or network process alive.
|
||||
- `scripts/peer-assisted-benchmark.ts` is not evidence for this issue: its
|
||||
synthetic canvas source is timer-driven and its browser launch explicitly
|
||||
disables background timer, occluded-window, and renderer backgrounding.
|
||||
@@ -187,7 +185,8 @@ All sources were checked on 2026-08-24.
|
||||
|
||||
## Bounded Current-Browser Screening
|
||||
|
||||
On 2026-08-24, Chrome `151.0.7922.174` on Windows 25H2 build `26200.9168`
|
||||
On 2026-08-24, before the current motion-hint decision, Chrome
|
||||
`151.0.7922.174` on Windows 25H2 build `26200.9168`
|
||||
captured a continuously changing native WPF window through real
|
||||
`getDisplayMedia()`. The source remained visible and changing; capture returned
|
||||
1186x712 at a 30 fps setting. One VP8/libvpx sender used no video hint, a 5 Mbps
|
||||
@@ -250,8 +249,8 @@ them directly.
|
||||
Open `chrome://webrtc-internals` on Host and Viewer before sharing and keep its
|
||||
overhead constant across every run. Leave audio-debug recording, event-log
|
||||
recording, packet capture, and media recording disabled. The current Screener
|
||||
diagnostic download may be taken at state boundaries as a latest-sample
|
||||
cross-check; it is not used as a time series. If a stable reproduction suggests
|
||||
connection-details view may be used as a latest-sample cross-check; it is not a
|
||||
time series and has no diagnostic download. If a stable reproduction suggests
|
||||
freezing or discard, inspect the Host tab from a pre-opened separate control
|
||||
window in `chrome://discards` or `edge://discards` outside the timed measurement
|
||||
window. That page may expose URLs and therefore remains local evidence.
|
||||
|
||||
@@ -1,551 +1,127 @@
|
||||
# Browser Screen-Audio Quality Controls
|
||||
|
||||
Accessed: 2026-08-25
|
||||
|
||||
Status: current Browser source and production use strict `screener-v12`, fixed
|
||||
VP8, no video hint, and one Opus screen-audio path with bounded 64/128/256 kbps
|
||||
sender ceilings. Exact source and production identities are owned by
|
||||
[status](../status.md). The capture request disables local speech processing,
|
||||
live mutation has applied readback, relay track changes preserve the persistent
|
||||
stream, and SFU reconnect reacquires exact publications/senders.
|
||||
Exact-production `bf32859` physical evidence closes the active-source `1 kbps` report;
|
||||
audible game quality and real-game SFU A/V synchronization remain open.
|
||||
|
||||
## Observed-Bitrate Identity Boundary
|
||||
|
||||
The interval calculation `(bytes * 8) / milliseconds` already returns kbps, and
|
||||
the UI rounds that observed value rather than displaying a configured ceiling.
|
||||
Current source binds ordinary and SFU publisher stats to the exact active audio
|
||||
sender, merges the SFU sender reports, resets identity-changing baselines, keeps
|
||||
late relay audio changes on exact children, and reacquires exact SFU senders on
|
||||
reconnect. Explicit no-audio remains unknown.
|
||||
|
||||
Pinned LiveKit 2.22.0 already republishes screen tracks after a full reconnect
|
||||
from the saved publication options, including the audio preset; a signal-only
|
||||
resume does not replace the sender. Screener must still reacquire its own exact
|
||||
publication/sender references, reset identity-bound stats, fail closed when an
|
||||
expected republish is missing, and propagate same-stream audio changes to its
|
||||
downstream peer children. Reapplying unchanged audio sender parameters after
|
||||
every `Reconnected` event duplicates the SDK on stable reconnects. A later
|
||||
cleanup may skip that write when the sender identity and desired readback already
|
||||
match, while retaining saved-option updates and the latest-profile race check.
|
||||
|
||||
A Chrome 151 gate against exact production `bf32859` used real
|
||||
`getDisplayMedia()` with generated active audio. Two Host-direct children, one
|
||||
browser-relay child and one SFU Viewer all advanced audio energy and video frames
|
||||
at each ceiling and after source replacement; the active UI never remained at
|
||||
`1 kbps`. Direct/browser-relay traffic tracked the selected ceiling. SFU wire
|
||||
traffic exceeded the 64/128 ceilings because the accepted pinned LiveKit
|
||||
publication retains RED; this does not change the exact sender parameter
|
||||
readback. Ordinary-route audio-minus-video playout timestamps ranged from -101
|
||||
to +4 ms once RTCP mapping arrived. Chrome exposed no such field on the SFU
|
||||
subscription, which remains unknown rather than zero or inferred. A muted or
|
||||
idle Opus stream may still legitimately use about 1 kbps.
|
||||
|
||||
## Scope And Decision
|
||||
|
||||
This review covers audio returned with `getDisplayMedia()` and sent through a
|
||||
browser `RTCPeerConnection`. It does not decide whether the product ultimately
|
||||
needs Windows per-application capture; that remains a separate native-sender
|
||||
requirement.
|
||||
|
||||
The product keeps one screen-media audio mode and exposes only three bounded
|
||||
sender ceilings rather than codec plumbing:
|
||||
|
||||
- request audio with echo cancellation, noise suppression, automatic gain, and
|
||||
voice isolation disabled, while preferring two capture channels;
|
||||
- request window audio for a selected window and offer system audio for a full
|
||||
display;
|
||||
- retain `contentHint = "music"` only as source intent, with no quality claim;
|
||||
- treat an audio track as optional and warn before publishing when none exists;
|
||||
- preserve returned audio through source changes, picture pause, P2P, relay, and
|
||||
configured SFU routes;
|
||||
- offer 64/128/256 kbps sender ceilings, default 128 kbps, with one 256 kbps
|
||||
Opus receive maximum and DTX off across Web peer and SFU routes;
|
||||
- let the browser's existing WebRTC congestion control reduce actual audio and
|
||||
video traffic; do not add an application audio adaptation loop;
|
||||
- expose no channel-count, sample-rate, codec, stereo, DTX, RED, FEC, arbitrary
|
||||
bitrate slider, or application-owned adaptation control; and
|
||||
- expose local, read-only audio RTP diagnostics without treating negotiated
|
||||
codec fields as source-quality facts.
|
||||
|
||||
The music hint is metadata, not a codec or quality mode. Chromium accepts and
|
||||
reads it back, but the inspected libwebrtc audio-track interface has no matching
|
||||
content-hint input to the Opus encoder. Screener therefore does not credit the
|
||||
hint with a quality change. RFC 7587 and the pinned LiveKit presets, rather than
|
||||
an ordinary-PC benchmark, define the 128 kbps default: the RFC places full-band
|
||||
stereo music in a 64--128 kbps sweet spot, and LiveKit 2.22.0 names 128 kbps
|
||||
`musicHighQualityStereo`. Requested, applied, negotiated, and observed states
|
||||
remain separate.
|
||||
|
||||
The advanced panel is named Share advanced settings. Current source stores the
|
||||
64/128/256 choice and applies that sender `maxBitrate` ceiling on the existing
|
||||
Opus path when it creates a sender. A change during an active share serializes
|
||||
`getParameters()`/`setParameters()` updates and readback, keeps media and the old
|
||||
applied ceiling on failure, and does not claim room-wide
|
||||
convergence without a remote applied acknowledgement. This does not require
|
||||
audio codec renegotiation. On the declared Chrome/Edge screen-audio Host
|
||||
baseline, pinned LiveKit 2.22.0 can update an existing audio sender without
|
||||
republish. Firefox's initial publish path may also write the preset into Opus
|
||||
fmtp, so a sender-only increase is not claimed there until real readback and
|
||||
receive evidence pass. A future microphone/voice feature remains a separate
|
||||
track and processing path with its own
|
||||
AEC/noise-suppression/DTX contract. It must not turn movie or game audio into a
|
||||
voice-processed source.
|
||||
|
||||
## Deployed Reports And Echo Boundary
|
||||
|
||||
Users first reported that production
|
||||
`6ccb516a47261054f91dfa2fafa408d39ced59fc` sounded poor for movie/video screen
|
||||
audio. Current source and production use peer/SFU stereo and the matching
|
||||
`maxaveragebitrate=128000`, but the speech-gated sound remained. A
|
||||
phone Viewer connected directly through the SFU reproduced it, so browser-relay
|
||||
decode/re-encode is a route-specific amplifier rather than the common cause.
|
||||
The shared Host capture path runs before direct P2P, SFU and peer-relay routes.
|
||||
|
||||
The earlier delayed self-echo report on `769de201f7cc` has a different boundary:
|
||||
a viewer's voice is rendered by a separate voice application on the Host,
|
||||
enters the system mix, and is captured and sent back. It is source-content
|
||||
leakage, not ordinary microphone acoustic echo. AEC, noise suppression, or
|
||||
post-mix Web Audio cannot recover the originating process and must not be
|
||||
presented as a fix.
|
||||
|
||||
On 2026-08-21 the production picker was user-verified to keep tab and window
|
||||
audio scoped as intended in that browser environment. Current source requests
|
||||
the source-appropriate window and full-display audio choices. This is one
|
||||
environment observation, not a cross-browser source guarantee.
|
||||
|
||||
## Voice-Processing Boundary
|
||||
|
||||
Extension `tabCapture`/`desktopCapture` tests do not establish speech-processing
|
||||
defaults for the Web Screen Capture API. Chromium M142 classifies ordinary web
|
||||
`getDisplayMedia()` as
|
||||
`kOther`, not `kExtensionScreenShare`. Without explicit constraints, that path
|
||||
selects browser-decided echo cancellation and defaults noise suppression and
|
||||
automatic gain control on. Its processed candidate also defaults to one channel.
|
||||
|
||||
This is real source processing, not just settings metadata:
|
||||
`MakeForDisplayCapture()` creates a local WebRTC audio-processing module and
|
||||
uses PeerConnection playout as the echo reference. Setting `contentHint` to
|
||||
`music` after capture does not rewrite those source properties; current Blink's
|
||||
WebRTC audio sink has no matching handler, and libwebrtc's Opus encoder retains
|
||||
a TODO for content-hint use.
|
||||
|
||||
The Chrome/Edge capture request therefore supplies bare `false` constraints for
|
||||
echo cancellation, noise suppression, automatic gain, and voice isolation, plus
|
||||
`channelCount: { ideal: 2 }`. These are post-selection preferences, not source
|
||||
picker restrictions or cross-browser guarantees. The declared Host baseline
|
||||
accepts them; a browser may ignore unsupported dictionary members. No `exact`,
|
||||
`min`, `advanced`, or fixed sample rate is requested, so this change does not
|
||||
turn optional screen audio into a hard capture gate.
|
||||
|
||||
## Standard API Boundary
|
||||
|
||||
| Concern | Standard request or control | Honest readback | What it does not prove | Current product action |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| Audio presence | A `MediaTrackConstraints` audio dictionary expresses interest | `stream.getAudioTracks().length` proves only whether a track was returned | The browser may still return video only; a track does not identify system, window, tab, or selected-game audio | Keep the existing presence check and visible no-audio warning |
|
||||
| Audio source choice | `windowAudio: "window"` asks for window audio and `systemAudio: "include"` offers system audio for monitor surfaces; a user agent may ignore either hint | There is no standard audio-source category readback corresponding to those hints | The selected audio scope, per-application isolation, or cross-browser availability | Let the picker expose the source-appropriate option without adding an inferred source label |
|
||||
| Content intent | `track.contentHint = "music"` records the source intent | The assigned hint can be read back | A codec, bitrate, stereo mode, or evidence that current Chromium changed Opus encoding | Keep it as metadata only; do not use it as a quality acceptance signal |
|
||||
| Voice-app exclusion | `restrictOwnAudio` concerns audio produced by the document that invoked capture; `suppressLocalAudioPlayback` concerns local playback of a captured browser surface | The app may observe whether a returned track exists, not which OS processes it contains | Excluding Discord, KOOK, WeChat, notifications, or any other independent process from system audio | Web keeps video-only available and warns that system audio may include calls/notifications; it does not claim isolation |
|
||||
| Source processing | Current Chromium accepts EC/NS/AGC/voice-isolation constraints on web display audio | Corresponding `track.getSettings()` fields can confirm values when exposed | Portable support, or that a missing field means false | Request all four off for movie/game audio and verify the target Host settings |
|
||||
| Capture channels | Current Chromium accepts `channelCount: { ideal: 2 }`; the Screen Capture specification does not make it portable | `track.getSettings().channelCount` may describe the returned track when exposed | That every source/browser can supply stereo, or that RTP sends stereo | Prefer two without using `exact`; keep negotiated Opus truth separate |
|
||||
| Capture sample rate | The Screen Capture specification does not list generic `sampleRate` as applicable to display audio | `track.getSettings().sampleRate` may be present in an implementation | The Opus mode, RTP clock semantics, receiver output rate, or end-to-end fidelity | Observe only in a future diagnostic; absent means unknown |
|
||||
| RTP send bitrate | `RTCRtpSender.setParameters()` can set `encodings[].maxBitrate` for audio; Opus `maxaveragebitrate` advertises the receiver's maximum | A following `getParameters()` can show the applied sender ceiling; negotiated fmtp shows the receiver maximum; outbound byte deltas show actual traffic | A minimum, audible improvement, or moment-to-moment rate; congestion may keep traffic lower | Apply the selected 64/128/256 kbps sender ceiling, advertise one 256 kbps receive maximum, and retain actual bitrate as the acceptance fact |
|
||||
| Codec | The user agent chooses among negotiated send codecs unless a separately negotiated codec selection is available | `RTCCodecStats.mimeType` and `sdpFmtpLine` identify the codec and negotiated format parameters in use | That Opus was selected before stats exist, or that negotiated preferences describe actual content | Label only observed negotiated data; do not force a codec |
|
||||
| Stereo | No stable sender parameter controls Opus stereo; RFC 7587 defines `stereo` as the receiver's one-way preference | Capture settings and negotiated codec/fmtp data can be inspected separately | Capture channel count and `opus/48000/2` do not prove encoded stereo | Use one bounded Opus answer transform on the declared Chrome/Edge baseline; no user control or inferred badge |
|
||||
| DTX | The current WebRTC `RTCRtpEncodingParameters` dictionary has no `dtx` member; RFC 7587 defaults absent `usedtx` to `0` | No portable standard sender readback proves DTX operation | A deprecated browser field or SDK option is not a cross-browser contract | Keep peer default-off and set LiveKit `dtx: false`; no user control |
|
||||
| FEC | There is no direct, portable audio FEC on/off sender parameter | Negotiated codec/fmtp data and inbound `fecPacketsReceived`/`fecBytesReceived` may provide evidence after use | Negotiation does not prove recovery occurred; zero counters do not prove FEC was disabled | No control; retain stats only as future diagnostic evidence |
|
||||
|
||||
The WebRTC specification defines audio `maxBitrate` as a maximum and allows
|
||||
other limits to constrain the sender further. A retained Chrome 151 loopback
|
||||
showed only that the browser applied `maxBitrate` and changed its encoder budget
|
||||
for that synthetic fixture; the ordinary PC, short interval and synthetic
|
||||
source are not calibration evidence. RFC 7587's full-band stereo music range
|
||||
and the pinned LiveKit preset justify the product value. Audible quality,
|
||||
actual traffic and congestion behavior remain runtime observations.
|
||||
|
||||
## Capture And Browser Compatibility
|
||||
|
||||
The Screen Capture Working Draft says constraints are applied only after the
|
||||
user chooses a surface and that `getUserMedia()` constraints do not apply to
|
||||
display tracks unless this specification lists them. The only portable audio
|
||||
properties it currently lists are `restrictOwnAudio` and
|
||||
`suppressLocalAudioPlayback`; neither controls fidelity. Chromium's processing
|
||||
and channel behavior is therefore an implementation contract for the declared
|
||||
Chrome/Edge Host baseline. The specification also explicitly permits a browser
|
||||
to return no audio despite an audio request and rejects `advanced` or `min`/
|
||||
`exact` display constraints; this request uses none of them.
|
||||
|
||||
MDN browser-compatibility data on the access date reports:
|
||||
|
||||
- display-audio capture in Chromium from Chrome 74, with whole-system audio on
|
||||
Windows and ChromeOS but tab-only audio on Linux and macOS;
|
||||
- no display-audio capture support in Firefox or Safari;
|
||||
- `systemAudio` in desktop Chromium from Chrome 105, but not Firefox or Safari;
|
||||
- `windowAudio` is listed as partial in Chrome 141: `"exclude"` and `"system"`
|
||||
are documented while `"window"` is not yet represented as supported in BCD;
|
||||
current Chrome deployments may differ, so runtime/user evidence stays local;
|
||||
and
|
||||
- `MediaTrackSettings.channelCount` as non-Baseline because it is missing in
|
||||
widely used browsers.
|
||||
|
||||
These are an implementation snapshot, not permanent product guarantees.
|
||||
Chromium's own capture documentation also separates Windows/ChromeOS system
|
||||
loopback capture from tab capture. It publishes no stable Web contract for a
|
||||
fixed display-audio sample rate or channel count.
|
||||
|
||||
`encodings[].maxBitrate` has much broader compatibility: MDN data records
|
||||
Chrome 69, Firefox 46, and Safari 11. The old `encodings[].dtx` field is absent
|
||||
from Chrome and Firefox, present only as a deprecated non-standard Safari
|
||||
feature, and absent from the current WebRTC IDL.
|
||||
|
||||
## Opus Signaling Is Not Source Truth
|
||||
|
||||
WebRTC endpoints must implement Opus, but negotiation may select another
|
||||
mandatory audio codec. The application must wait for the in-use codec stats
|
||||
before calling a stream Opus.
|
||||
|
||||
RFC 7587 requires an Opus SDP `a=rtpmap` clock rate of 48000 and channel count
|
||||
of 2 for all Opus sessions. It separately explains that actual Opus media may
|
||||
use another internal sampling rate. Therefore `audio/opus`, `clockRate=48000`,
|
||||
and `channels=2` in codec stats do not establish a 48 kHz stereo capture or
|
||||
stereo payload.
|
||||
|
||||
Opus `maxaveragebitrate`, `stereo`, `useinbandfec`, and `usedtx` are fmtp
|
||||
offer/answer parameters. `stereo` is a receive preference, while
|
||||
`sprop-stereo` is only a sender hint and explicitly not a guarantee. A browser
|
||||
API that merely exposes the negotiated `sdpFmtpLine` can report those facts,
|
||||
but does not provide a portable setter or prove the encoder's moment-to-moment
|
||||
behavior.
|
||||
|
||||
Current libwebrtc source calculates the default full-band Opus bitrate as
|
||||
32 kbps times the negotiated channel count. It derives two encoded channels
|
||||
only from `stereo=1`, so stereo without `maxaveragebitrate` starts from a
|
||||
64 kbps codec bitrate. `RTCRtpSender.maxBitrate=128000` only caps the allocator;
|
||||
it does not raise that codec configuration. When present, libwebrtc maps the
|
||||
negotiated `maxaveragebitrate` value into the Opus encoder bitrate config before
|
||||
normal congestion adaptation. This is implementation evidence for the current
|
||||
Chrome/Edge baseline, while the RFC still defines the parameter as a receiver
|
||||
maximum rather than a minimum or continuous bitrate guarantee.
|
||||
|
||||
The poor movie/music report against production `6ccb516` began under a
|
||||
historical mono negotiation contract, but it remained after a later 128-only
|
||||
source stage added peer `stereo=1;maxaveragebitrate=128000` and SFU
|
||||
`forceStereo: true`, 128 kbps, DTX off, and RED retained. Current Share advanced
|
||||
settings provide 64/128/256 kbps sender ceilings with 128 kbps as the default;
|
||||
those encoder settings still cannot restore PCM
|
||||
already changed or collapsed to mono by the Host capture processor. Loss,
|
||||
jitter and concealment remain separate diagnostic facts, but direct SFU
|
||||
reproduction makes source processing the first reversible correction.
|
||||
|
||||
## Reference Implementations
|
||||
|
||||
No code was copied. All repositories were inspected at pinned commits on
|
||||
2026-08-19 and their cited source pages were rechecked on 2026-08-21.
|
||||
|
||||
- LiveKit client-sdk-js 2.22.0, Apache-2.0, commit
|
||||
[`0a2110d`](https://github.com/livekit/client-sdk-js/tree/0a2110d39904a06722a0c4d1ddbb9390bb06ad4d):
|
||||
it treats
|
||||
[`getSettings().channelCount` or requested constraints](https://github.com/livekit/client-sdk-js/blob/0a2110d39904a06722a0c4d1ddbb9390bb06ad4d/src/room/participant/LocalParticipant.ts#L899-L920)
|
||||
as a stereo-input heuristic and populates standard
|
||||
`RTCRtpEncodingParameters.maxBitrate`, but stereo/DTX/RED also travel in
|
||||
LiveKit signaling. Its transport then
|
||||
[munges Opus bitrate fmtp](https://github.com/livekit/client-sdk-js/blob/0a2110d39904a06722a0c4d1ddbb9390bb06ad4d/src/room/PCTransport.ts#L269-L333)
|
||||
in SDP. Its room defaults are music 48 kbps, DTX on, RED on and
|
||||
`forceStereo: false`; its named presets include stereo music at 64 kbps and
|
||||
high-quality stereo music at 128 kbps. Per-track options merge after those
|
||||
defaults. Its stereo-only `??=` fallback disables DTX/RED only if still
|
||||
undefined, which the pinned room defaults are not. Screener's historical
|
||||
pre-stereo source overrode DTX but not `forceStereo` or RED, yielding mono,
|
||||
DTX off and RED on.
|
||||
- lib-jitsi-meet, Apache-2.0, commit
|
||||
[`63a04ec`](https://github.com/jitsi/lib-jitsi-meet/tree/63a04ecabd972ea75e877f9ba12086c13cb68210):
|
||||
[`mungeOpus()`](https://github.com/jitsi/lib-jitsi-meet/blob/63a04ecabd972ea75e877f9ba12086c13cb68210/modules/RTC/TPCUtils.ts#L871-L929)
|
||||
writes `stereo`, `sprop-stereo`, `maxaveragebitrate`, and `usedtx` into SDP,
|
||||
including a Firefox-specific DTX exception.
|
||||
- simple-peer, MIT, commit
|
||||
[`f1a492d`](https://github.com/feross/simple-peer/tree/f1a492d1999ce727fa87193ebdea20ac89c1fc6d):
|
||||
it has no higher-level audio quality contract and exposes only an advanced
|
||||
generic
|
||||
[`sdpTransform`](https://github.com/feross/simple-peer/blob/f1a492d1999ce727fa87193ebdea20ac89c1fc6d/index.js#L608-L664)
|
||||
hook for SDP changes.
|
||||
|
||||
The Apache and MIT implementations support the same conclusion: one narrow,
|
||||
structured Opus fmtp transform is established practice, but it is still an
|
||||
application-owned negotiation contract rather than a portable sender setter.
|
||||
`sdp-transform` 2.15.0 is MIT-licensed and already arrives transitively through
|
||||
LiveKit. Runtime code that imports it must declare it directly, together with
|
||||
its TypeScript types, rather than depend on LiveKit's dependency graph.
|
||||
As supplementary practice evidence, historical
|
||||
[LiveKit server issue 970](https://github.com/livekit/livekit/issues/970) and
|
||||
the 2026 [Rust SDK issue 1018](https://github.com/livekit/rust-sdks/issues/1018)
|
||||
both report stereo sources arriving as mono when the product/SDK did not signal
|
||||
stereo. Issue reports are not treated as normative behavior or as proof of
|
||||
Screener's runtime state; pinned source and local stats remain authoritative.
|
||||
|
||||
Discord and Oopz have a narrower evidence boundary:
|
||||
|
||||
- Discord's official Go Live architecture says its native client captures the
|
||||
selected process and child-process audio with OS-specific APIs and sends
|
||||
audio and video in separate RTP packets. Discord's public Voice protocol
|
||||
requires Opus at the protocol boundary and distinguishes a `Soundshare`
|
||||
speaking bit, but neither source publishes the exact negotiated/runtime
|
||||
audio parameters for each consumer Go Live session. This is useful native
|
||||
product evidence, not a preset Screener can copy or an excuse to infer codec
|
||||
state without stats.
|
||||
- Oopz's official help separates program-audio screen sharing from voice
|
||||
channels: it documents Windows program-audio capture, while separate voice
|
||||
articles expose channel quality/latency choices and microphone AI noise
|
||||
suppression. The retained read-only
|
||||
[Oopz 0.87.425 package inspection](./native-shared-encode-sender.md) found a
|
||||
native Agora RTC library, program/system screen-audio switches, and Agora
|
||||
profile symbols including high-quality stereo music and game-streaming
|
||||
scenarios. This proves only separate native capture/voice/profile surfaces.
|
||||
Neither the package nor public help proves the active screen-share codec,
|
||||
bitrate, stereo, DTX, FEC or Agora profile; those values remain unknown.
|
||||
|
||||
## Honest Diagnostics Contract
|
||||
|
||||
Audio diagnostics keep four layers separate:
|
||||
|
||||
1. **Capture:** track absent/present, plus `channelCount` and `sampleRate` only
|
||||
when the browser returns numeric values.
|
||||
2. **Requested/applied:** an RTP ceiling only if Screener actually calls
|
||||
`setParameters()`, followed by immediate `getParameters()` readback.
|
||||
3. **Negotiated:** current local details follow the unique audio RTP object's
|
||||
`codecId` to a same-transport `RTCCodecStats` and display MIME type, clock
|
||||
rate, channels and bounded fmtp as negotiated fields. Raw SDP/stats/fmtp are
|
||||
not uploaded or persisted.
|
||||
4. **Observed:** current local details derive interval audio bitrate from byte
|
||||
deltas and interval loss from `lostDelta / (receivedDelta + lostDelta)`, and
|
||||
display reported jitter. For the unique current inbound screen video/audio
|
||||
pair, they also display audio-minus-video `estimatedPlayoutTimestamp`,
|
||||
interval average jitter-buffer delay as
|
||||
`delta(jitterBufferDelay) / delta(jitterBufferEmittedCount)`, audio concealed
|
||||
samples as `delta(concealedSamples) / delta(totalSamplesReceived)`, and the
|
||||
interval `concealmentEvents` count. RTP stats id, SSRC or track-identifier
|
||||
changes rebase that media kind; a counter reset rebases the affected
|
||||
interval. The first sample, negative delta, zero denominator, ambiguous RTP
|
||||
object or absent field is unknown. These local-only fields are not added to
|
||||
Viewer quality-evidence signaling or persistence. Browser availability and
|
||||
actual route-switch synchronization remain target-device evidence; FEC
|
||||
counters remain an acceptance-matrix follow-up.
|
||||
|
||||
Never substitute `48000`, `2`, `false`, or `0` for an unavailable field.
|
||||
|
||||
## Minimal Route-Consistent Runtime Slice
|
||||
|
||||
The route-consistent runtime covers capture and negotiation without replacing
|
||||
browser congestion control:
|
||||
|
||||
1. Request display audio with echo cancellation, noise suppression, automatic
|
||||
gain, and voice isolation disabled plus ideal two-channel capture. Keep
|
||||
`contentHint = "music"` as metadata and one audio track. Apply the selected
|
||||
64/128/256 kbps sender ceiling, defaulting missing/legacy state to 128 kbps.
|
||||
Do not force a sample rate or add Web Audio mixing,
|
||||
resampling, a second representation or a new rate controller.
|
||||
2. On every Web Viewer answer, including ICE restart/rebuild answers, parse SDP
|
||||
with direct dependencies on MIT-licensed `sdp-transform` 2.15.0 and its
|
||||
TypeScript declarations. Select the single non-rejected audio media section,
|
||||
resolve its Opus payload from structured `rtp` entries, and idempotently
|
||||
upsert `stereo=1` plus `maxaveragebitrate=256000` in that payload's fmtp before both
|
||||
`setLocalDescription()` and signaling the same description. Preserve every
|
||||
other media section, codec, fmtp key and session attribute. Empty SDP, no
|
||||
unambiguous active audio/Opus payload, multiple active audio sections,
|
||||
malformed fmtp, or parser/write failure returns the original answer object
|
||||
and SDP unchanged. This keeps a browser-generated mono/video answer usable;
|
||||
no broad regex or malformed partial answer is allowed. RFC 7587 makes stereo
|
||||
an optional receive preference, while W3C guarantees the unmodified
|
||||
`createAnswer()` remains usable by `setLocalDescription()`.
|
||||
3. Do not add `sprop-stereo`: RFC 7587 defines it as a sender-likelihood hint,
|
||||
not the receiver preference that permits the remote encoder to send stereo,
|
||||
and offer/answer parameters are orthogonal. Do not add `usedtx=0`; absent
|
||||
`usedtx` already means off. `maxaveragebitrate=256000` is the receiver maximum
|
||||
covering all three sender ceilings, not a minimum or current target; stock
|
||||
congestion control can still lower actual traffic. Keep the browser's
|
||||
existing `useinbandfec` negotiation untouched.
|
||||
4. For SFU publication map 64 kbps to LiveKit 2.22.0
|
||||
`AudioPresets.musicStereo`, 128 kbps to `musicHighQualityStereo`, and 256 kbps
|
||||
to an explicit preset; retain `forceStereo: true` and `dtx: false`.
|
||||
Do not override the pinned room default `red: true`. RED and Opus in-band FEC
|
||||
are distinct repair mechanisms; retaining one neither proves, replaces nor
|
||||
disables the other, and no FEC switch is exposed or reimplemented.
|
||||
|
||||
Applying the receive preference in every browser answer covers direct P2P and
|
||||
browser-relay children. SFU uses the equivalent SDK contract.
|
||||
An SFU-only change is not acceptable: moving between peer and SFU routes would
|
||||
otherwise change channel behavior. Native/Pion sender routes retain their own
|
||||
explicit encoded-track contract and need a focused compatibility check before
|
||||
claiming the same result.
|
||||
|
||||
At the default ceiling, audio contributes at most 128 kbps RTP payload per
|
||||
active outbound edge, or 256 kbps for the configured two-slot endpoint example.
|
||||
The endpoint cap itself follows the current route policy. The
|
||||
explicit highest preset doubles those bounds to 256/512 kbps, excluding
|
||||
RTP/SRTP/UDP/IP overhead. A Host SFU publication contributes one such ingress;
|
||||
each root/peer forward still sends its own copy. This is small beside the
|
||||
current 3--8 Mbps video presets, but stereo can use more encoder work and
|
||||
bandwidth than mono. No second track or representation is added, and browser
|
||||
relays still make no shared-encoder promise.
|
||||
|
||||
Preserve the existing persistent Viewer `MediaStream`/media element and add or
|
||||
remove its audio/video tracks in place. Introducing a separate audio element or
|
||||
Web Audio graph would create a new clock/buffering boundary and is outside this
|
||||
fix; the stable stream already deployed is the A/V synchronization baseline.
|
||||
The local receiver diagnostic observes only `getStats()` on that existing
|
||||
route. It does not set `jitterBufferTarget`, create a clock, modify playout, add
|
||||
a wire field, or claim that a missing browser statistic is zero.
|
||||
|
||||
The implementation budget is deliberately bounded to dependency
|
||||
manifest/lockfile entries, one SDP helper, `ViewerPeer` answer wiring, SFU
|
||||
publisher options, focused unit/integration tests and the owning docs. Parser
|
||||
tests cover CRLF serialization, an existing/missing fmtp line, exact stereo and
|
||||
bitrate replacement, unknown fmtp preservation, idempotence, payload-number
|
||||
lookup, rejected/non-audio sections, and identity-preserving fallback for empty
|
||||
SDP, no/ambiguous Opus, duplicate targets, and parser exceptions. Peer tests
|
||||
prove the exact selected answer is both applied and
|
||||
signaled on initial/restart/rebuild paths; SFU tests assert the three explicit
|
||||
options and absence of a RED override. One Chrome/Edge left/right fixture checks
|
||||
channel correctness and a P2P/SFU route switch checks the persistent stream;
|
||||
neither is a performance benchmark or parameter-calibration gate.
|
||||
|
||||
## Native Windows Process-Audio Candidate
|
||||
|
||||
The smallest honest native path is Windows WASAPI application loopback through
|
||||
`ActivateAudioInterfaceAsync` and `AUDIOCLIENT_PROCESS_LOOPBACK_PARAMS` with
|
||||
`INCLUDE_TARGET_PROCESS_TREE`. After the user selects a game window, the sender
|
||||
resolves its PID locally and captures that process plus children. The API's
|
||||
minimum build 20348 is not available on ordinary Windows 10 22H2 (19045), so
|
||||
the current product candidate covers Windows 11 only. Windows 10 game-only
|
||||
audio remains unresolved or explicitly unsupported until another bounded path
|
||||
passes.
|
||||
|
||||
Default native behavior is game-only. Whole-system audio requires an explicit
|
||||
choice. No render stream, process exit, or capture failure produces a visible
|
||||
silent/failed state and never widens capture to the whole system. Excluding one
|
||||
known voice process is not a sufficient default because other calls,
|
||||
notifications, or browser audio could still leak. Protected content may be
|
||||
absent and an exclusive/no-render path may yield silence rather than a
|
||||
classifiable error; the sender reports silence or unknown and never bypasses
|
||||
protection. PID, path, window title, device identity, and raw audio remain local
|
||||
and are not retained or uploaded.
|
||||
|
||||
### P1 Audio-Only Wiring Slice
|
||||
|
||||
The first native implementation is intentionally an audio-only input and is
|
||||
estimated at 500--800 new lines. It keeps the current Web video capture and
|
||||
codec path, while a Windows 11 helper supplies process-tree PCM through the
|
||||
existing authenticated local bridge. Chrome/Edge's already-required WebCodecs
|
||||
surface encodes that PCM with `AudioEncoder` (`opus`, 48 kHz, two channels,
|
||||
20 ms), so the helper does not add a native Opus dependency or a second package
|
||||
toolchain. The Go sender packetizes the returned Opus access units once into a
|
||||
shared `audio/opus` RTP track before adding it to each native PeerConnection.
|
||||
|
||||
The local envelope carries only media kind, flags, QPC-derived timestamp,
|
||||
duration, and bounded payload; no PID or source label crosses the bridge. WGC
|
||||
video remains a fixture/next adapter in this slice. Its
|
||||
`SystemRelativeTime`, like WASAPI `GetBuffer`'s `pu64QPCPosition`, is a 100-ns
|
||||
QPC value and is retained as the future A/V synchronization origin. Audio
|
||||
timestamps are converted to the 48 kHz RTP clock and remain monotonic; packet
|
||||
duration comes from the captured frame count rather than arrival time.
|
||||
|
||||
This is an opt-in P1 candidate, not a default or production switch. Windows 10,
|
||||
picker/activation denial, an exited or inaccessible target, no render stream,
|
||||
protected-content silence, and WASAPI device/service errors report
|
||||
`unavailable`/`silent` and stop or ask. The implementation never widens to
|
||||
whole-system loopback. The first acceptance smoke is one direct Viewer with a
|
||||
known target tone/visual marker and independent voice/notification markers;
|
||||
SFU/UDP, a second Viewer, endurance, and native WGC video replacement remain
|
||||
later gates.
|
||||
|
||||
### Retained P1 Result (2026-08-21)
|
||||
|
||||
The source-complete Windows helper emits process-tree-only 48 kHz stereo s16 in
|
||||
20-ms chunks. Chrome 151 encodes the chunks once with `AudioEncoder`/Opus, and
|
||||
Go packetizes the same encoded stream once into a shared Pion track. The UI
|
||||
receives an opaque memory-only target ID. PID and the enumeration-time process
|
||||
creation token are revalidated locally and never enter public signaling, a URL,
|
||||
storage, or diagnostics.
|
||||
|
||||
One bounded run played 440 Hz in the selected target and 880 Hz in an
|
||||
independent process. Across 100 chunks, amplitudes were 0.240225 and 0.0000598
|
||||
(4017.8x); no system-mix fallback was used. One direct Viewer then received one
|
||||
audio track and 495 inbound Opus packets while video decoded/rendered 296
|
||||
frames. There was no fatal or encoder error. The sole failed assertion was the
|
||||
existing generic Pion two-second outbound snapshot, which also missed fresh
|
||||
deltas in earlier video loopbacks. This is functional evidence, not packaging,
|
||||
real-game sync, second-Viewer, SFU/UDP, or endurance evidence.
|
||||
|
||||
Run one bounded matrix rather than a full route Cartesian product against
|
||||
the exact production release and matching runtime source at execution time on
|
||||
Windows Chrome/Edge for tab/window/monitor,
|
||||
audio selected/unselected, and a simultaneous voice call; then the native
|
||||
candidate on current Windows 11 with game parent and child audio, an independent
|
||||
voice process, notifications, no render stream, and process restart. Windows 10
|
||||
records the explicit unsupported/unresolved result rather than a fake fallback.
|
||||
Direct is the primary route; one browser-relay and one SFU-subscription audio
|
||||
check cover route preservation. Ordinary peer ICE remains STUN-only and current
|
||||
SFU media remains UDP-only; this audio matrix does not add a transport path.
|
||||
Correlate capture settings, negotiated codec/derived fmtp,
|
||||
actual outbound/inbound bitrate, loss, jitter, concealment, jitter buffer, and
|
||||
A/V playout timing using a distinguishable stereo fixture plus game/film audio.
|
||||
|
||||
## UI And Voice Boundary
|
||||
|
||||
Share advanced settings offers exactly 64/128/256 kbps and defaults to 128.
|
||||
Current source applies the selected choice before sharing, to every newly
|
||||
created P2P, browser-relay, or SFU sender, and through a serialized live
|
||||
mutation; production deploys the same path. It reads
|
||||
the latest desired profile into each sender and keeps endpoint-local applied
|
||||
readback rather than claiming one room-wide applied commit. The setting remains
|
||||
a sender ceiling, not a guaranteed or constant bitrate. Do not expose sample
|
||||
rate, channel count, codec, DTX, RED, FEC, an arbitrary slider, or a second audio
|
||||
adaptation loop.
|
||||
|
||||
If microphone voice enters scope, treat it as a separate source/track with an
|
||||
independent privacy, AEC, noise-suppression, gain and DTX design. Oopz's public
|
||||
product surfaces support that separation but do not supply parameters that can
|
||||
be copied into Screener's screen-media path.
|
||||
# Browser Screen-Audio Quality
|
||||
|
||||
- Research date: 2026-08-25
|
||||
- Scope: Browser game/movie screen audio over P2P, relay, and LiveKit SFU
|
||||
- Status: current Browser contract implemented and deployed; heterogeneous
|
||||
devices and real-game A/V synchronization remain open
|
||||
|
||||
## Current Conclusion
|
||||
|
||||
Screener requests display audio by default on initial capture and source switch.
|
||||
If the browser, selected source, operating system, or user returns no audio
|
||||
track, video sharing continues with an explicit warning.
|
||||
|
||||
One screen-audio track uses `contentHint = "music"`. Capture requests ideal
|
||||
stereo and disables echo cancellation, noise suppression, automatic gain, and
|
||||
voice isolation where the browser supports those constraints. These are
|
||||
preferences and readback fields, not portable fidelity guarantees.
|
||||
|
||||
The only user-facing audio-quality choices are sender ceilings of 64, 128, and
|
||||
256 kbps, with 128 kbps as the default. They update live on current P2P,
|
||||
Browser-relay, and SFU senders and apply to future senders. Actual RTP traffic
|
||||
can remain below a ceiling because of content and congestion, and includes
|
||||
transport behavior not represented by the selected number.
|
||||
|
||||
## Peer Contract
|
||||
|
||||
Direct and Browser-relay Viewer answers use one structured `sdp-transform`
|
||||
operation on the single active Opus audio section. It idempotently requests
|
||||
`stereo=1` and `maxaveragebitrate=256000`, preserving every unrelated codec,
|
||||
format parameter, media section, and session attribute. Missing, ambiguous, or
|
||||
malformed targets return the browser-generated answer unchanged.
|
||||
|
||||
The 256 kbps receive maximum permits every sender ceiling; it is not a current
|
||||
target or minimum. Screener does not set `sprop-stereo`, rewrite FEC, force a
|
||||
sample rate, or add broad string/regular-expression SDP mutation.
|
||||
|
||||
Each audio sender reads fresh parameters, applies the selected `maxBitrate`, and
|
||||
reads the result back. A rejected update leaves the previous sender state and
|
||||
media alive. Without a remote applied acknowledgement, the Host does not claim
|
||||
room-wide atomic convergence.
|
||||
|
||||
## LiveKit Contract
|
||||
|
||||
The SFU publisher maps the three ceilings to pinned LiveKit stereo music presets
|
||||
or an explicit 256 kbps preset. Every SFU audio publication uses:
|
||||
|
||||
- `forceStereo: true`;
|
||||
- `dtx: false`; and
|
||||
- `red: false`.
|
||||
|
||||
RTP RED sends redundant copies of prior audio data to improve recovery from
|
||||
short packet-loss bursts. It also materially increases payload traffic; the
|
||||
earlier 128 kbps selection produced about 256 kbps on the measured SFU path.
|
||||
Screener disables RED so the selected audio ceiling remains a useful publisher
|
||||
budget. This trades away RED-specific loss resilience. Opus in-band FEC and
|
||||
other WebRTC recovery remain separately negotiated; disabling RED neither proves
|
||||
nor disables them.
|
||||
|
||||
LiveKit reconnect reacquires the exact current publication and sender before
|
||||
reapplying audio parameters. Unchanged publication/sender identity does not
|
||||
reset stats or parameter state. Missing current ownership fails the SFU route
|
||||
closed instead of mutating a stale sender.
|
||||
|
||||
## Relay And Presentation
|
||||
|
||||
A Browser relay keeps one persistent `MediaStream` and reconciles screen-audio
|
||||
track addition, removal, and replacement into each current child sender. A
|
||||
source switch does not create an independent audio clock or Web Audio graph.
|
||||
|
||||
Viewer playback uses the browser's native media controls on the one persistent
|
||||
video element. Local volume, mute, play/pause, and fullscreen do not enter
|
||||
signaling, sender configuration, room state, or another participant. Autoplay
|
||||
rejection remains a typed presentation state until the browser receives a user
|
||||
gesture.
|
||||
|
||||
Connection metrics select the exact current audio sender/receiver. When audio is
|
||||
active, the UI reports actual interval traffic rather than a constant or
|
||||
placeholder `1 kbps`. Track and sender replacement reset only the affected stats
|
||||
identity, so stale asynchronous samples cannot overwrite the new route.
|
||||
|
||||
## Evidence Boundary
|
||||
|
||||
Chrome 151 production gates exercised direct Host children, a Browser-relay
|
||||
child, and an SFU subscriber with generated screen audio. Video and decoded
|
||||
audio energy advanced at all three 64/128/256 kbps ceilings and after source
|
||||
replacement. Direct and Browser-relay traffic tracked the selected ceiling;
|
||||
the then-enabled SFU RED path explained the approximately doubled SFU audio
|
||||
payload.
|
||||
|
||||
The same evidence verified that the active-audio display no longer reports the
|
||||
old constant `1 kbps`. It did not establish perceptual music quality, all
|
||||
capture-source combinations, weak-network loss recovery after RED is disabled,
|
||||
or real-game SFU A/V synchronization.
|
||||
|
||||
## Boundaries
|
||||
|
||||
- No microphone/voice processing policy is inferred from screen audio. A future
|
||||
microphone is a separate source and track.
|
||||
- No user control for codec, stereo, channel count, sample rate, DTX, RED, FEC,
|
||||
or arbitrary bitrate.
|
||||
- No Web Audio mixing, resampling, second representation, or independent audio
|
||||
rate controller.
|
||||
- `opus/48000/2`, capture channel count, and configured ceilings do not prove
|
||||
encoded stereo or end-to-end fidelity.
|
||||
- Browser/system capture scope may include calls or notifications; the page
|
||||
cannot promise per-process isolation from generic system audio.
|
||||
|
||||
## Current Verification Gaps
|
||||
|
||||
- Chrome/Edge tab, window, and system-audio source combinations on supported
|
||||
Windows versions.
|
||||
- Real game/music subjective quality and A/V synchronization over direct,
|
||||
relay, and SFU paths.
|
||||
- Lossy Wi-Fi/cellular behavior with RED disabled, including concealment and
|
||||
Opus FEC counters where exposed.
|
||||
- macOS, Android, and iOS capture/playback behavior.
|
||||
|
||||
## Primary Sources
|
||||
|
||||
- [W3C Screen Capture](https://www.w3.org/TR/screen-capture/)
|
||||
- [W3C WebRTC](https://www.w3.org/TR/webrtc/)
|
||||
- [W3C WebRTC Priority Control](https://www.w3.org/TR/webrtc-priority/)
|
||||
- [W3C WebRTC Statistics](https://www.w3.org/TR/webrtc-stats/)
|
||||
- [W3C Web Audio](https://www.w3.org/TR/webaudio-1.1/)
|
||||
- [W3C WebCodecs Opus registration](https://www.w3.org/TR/webcodecs-opus-codec-registration/)
|
||||
- [libwebrtc Opus encoder](https://webrtc.googlesource.com/src/+/refs/heads/main/modules/audio_coding/codecs/opus/audio_encoder_opus.cc)
|
||||
- [libwebrtc media-track interface](https://webrtc.googlesource.com/src/+/refs/heads/main/api/media_stream_interface.h)
|
||||
- [Chromium `MediaStreamTrack` content hint](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/third_party/blink/renderer/modules/mediastream/media_stream_track_impl.cc)
|
||||
- [Chromium web display-audio constraint selection](https://github.com/chromium/chromium/blob/3620c35de32f20cfb11d0a616227c44750e31c67/third_party/blink/renderer/modules/mediastream/media_stream_constraints_util_audio.cc)
|
||||
- [Chromium M142 display-audio default restoration](https://chromium.googlesource.com/chromium/src/+/b059fa325c6da901f1b0b6afd9e736d67f62960e)
|
||||
- [Chromium display-capture audio processing](https://github.com/chromium/chromium/blob/0d07b03783490c156526384073fb5e97e7463e77/third_party/blink/renderer/modules/mediastream/media_stream_audio_processing_layout.cc)
|
||||
- [libwebrtc stats collector](https://webrtc.googlesource.com/src/+/refs/heads/main/pc/rtc_stats_collector.cc)
|
||||
- [LiveKit 2.22.0 track options](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/track/options.ts)
|
||||
- [LiveKit 2.22.0 publish defaults](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/defaults.ts)
|
||||
- [LiveKit 2.22.0 local publication](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/participant/LocalParticipant.ts)
|
||||
- [LiveKit 2.22.0 package dependencies](https://github.com/livekit/client-sdk-js/blob/v2.22.0/package.json)
|
||||
- [Jitsi Opus SDP transform, pinned commit](https://github.com/jitsi/lib-jitsi-meet/blob/63a04ecabd972ea75e877f9ba12086c13cb68210/modules/RTC/TPCUtils.ts)
|
||||
- [`sdp-transform` 2.15.0](https://github.com/clux/sdp-transform/tree/v2.15.0)
|
||||
- [RFC 7587: RTP Payload Format for Opus](https://www.rfc-editor.org/rfc/rfc7587.html)
|
||||
- [RFC 7874: WebRTC Audio Codec and Processing Requirements](https://www.rfc-editor.org/rfc/rfc7874.html)
|
||||
- [Discord Go Live architecture](https://discord.com/blog/how-it-all-goes-live-an-overview-of-discords-streaming-technology)
|
||||
- [Discord Voice protocol](https://docs.discord.com/developers/topics/voice-connections)
|
||||
- [Oopz program/screen audio help](https://help.oopz.cn/42f9/54e2)
|
||||
- [Oopz voice quality/latency help](https://help.oopz.cn/42f9/3020)
|
||||
- [Oopz microphone-noise help](https://help.oopz.cn/42f9/f5ae)
|
||||
- [MDN `getDisplayMedia()`](https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getDisplayMedia)
|
||||
- [MDN `MediaTrackSettings.channelCount`](https://developer.mozilla.org/en-US/docs/Web/API/MediaTrackSettings/channelCount)
|
||||
- [MDN Browser Compatibility Data: `MediaDevices`](https://github.com/mdn/browser-compat-data/blob/main/api/MediaDevices.json)
|
||||
- [MDN Browser Compatibility Data: `RTCRtpSender`](https://github.com/mdn/browser-compat-data/blob/main/api/RTCRtpSender.json)
|
||||
- [Chrome screen-sharing controls](https://developer.chrome.com/docs/web-platform/screen-sharing-controls)
|
||||
- [Chrome 141 window-audio release note](https://developer.chrome.com/release-notes/141)
|
||||
- [Chromium media-capture architecture index](https://chromium.googlesource.com/chromium/src/+/HEAD/docs/media/capture/)
|
||||
- [Microsoft Application Loopback sample](https://learn.microsoft.com/en-us/samples/microsoft/windows-classic-samples/applicationloopbackaudio-sample/)
|
||||
- [Microsoft `ActivateAudioInterfaceAsync`](https://learn.microsoft.com/en-us/windows/win32/api/mmdeviceapi/nf-mmdeviceapi-activateaudiointerfaceasync)
|
||||
- [Windows process-loopback parameters](https://learn.microsoft.com/en-us/windows/win32/api/audioclientactivationparams/ns-audioclientactivationparams-audioclient_process_loopback_params)
|
||||
- [Microsoft Windows Graphics Capture](https://learn.microsoft.com/en-us/windows/apps/develop/media-authoring-processing/screen-capture)
|
||||
- [Microsoft `IAudioCaptureClient::GetBuffer`](https://learn.microsoft.com/en-us/windows/win32/api/audioclient/nf-audioclient-iaudiocaptureclient-getbuffer)
|
||||
- [Windows 10 release information](https://learn.microsoft.com/en-us/windows/release-health/release-information)
|
||||
- [Microsoft WASAPI loopback recording](https://learn.microsoft.com/en-us/windows/win32/coreaudio/loopback-recording)
|
||||
- [Screen Capture](https://www.w3.org/TR/screen-capture/)
|
||||
- [Media Capture and Streams](https://www.w3.org/TR/mediacapture-streams/)
|
||||
- [WebRTC](https://www.w3.org/TR/webrtc/)
|
||||
- [RFC 7587: Opus RTP payload](https://www.rfc-editor.org/rfc/rfc7587.html)
|
||||
- [RFC 2198: RTP redundant audio](https://www.rfc-editor.org/rfc/rfc2198.html)
|
||||
- [LiveKit audio RED and media options](https://docs.livekit.io/transport/media/advanced/)
|
||||
- [LiveKit client 2.22.0 track options](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/track/options.ts)
|
||||
- [sdp-transform](https://github.com/clux/sdp-transform)
|
||||
@@ -1,834 +1,167 @@
|
||||
# Realtime Screen-Share Quality Adaptation
|
||||
|
||||
- Research date: 2026-08-25
|
||||
- Scope: realtime game screen sharing in the browser
|
||||
- Status: exact-production Browser gate completed; real-game and heterogeneous-device quality remain unverified
|
||||
- Scope: Browser game-screen capture, encoding, P2P forwarding, and LiveKit SFU
|
||||
- Status: current Browser policy accepted; real-game, weak-device, and
|
||||
heterogeneous-network quality remain open
|
||||
|
||||
## Finding
|
||||
## Current Conclusion
|
||||
|
||||
Screener should not optimize one quality dimension at the expense of the
|
||||
others. `maintain-framerate` may preserve motion by reducing resolution until
|
||||
game UI, maps, subtitles, and text become unreadable; `maintain-resolution`
|
||||
may instead lower frame rate. Neither preference overrides congestion control.
|
||||
Browser video is fixed VP8. Display video uses `contentHint = "motion"`, and
|
||||
display audio uses `contentHint = "music"`. The three recommended profiles and
|
||||
advanced settings are ceilings, not delivery guarantees. WebRTC owns direct and
|
||||
peer congestion control; LiveKit owns SFU representations, Dynacast, subscriber
|
||||
bandwidth estimation, and layer forwarding.
|
||||
|
||||
Production runs exact deployed application/runtime revision
|
||||
`be53c4d5794d38d5d406c876309958d7117ee601`, release `be53c4d`. Current Browser source and production use
|
||||
strict `screener-v12`, fixed VP8, no video `contentHint`, and no codec UI,
|
||||
quality state, or wire field. They use `balanced` as the recommended profile
|
||||
and advanced default;
|
||||
`maintain-resolution` and `maintain-framerate` remain explicit choices. These
|
||||
preferences leave actual degradation to the browser, so Screener observes
|
||||
readback and stats rather than claiming a fixed quality outcome.
|
||||
Screener does not define a resolution/FPS/bitrate ladder, custom SFU lower
|
||||
representation, scene detector, quality score, parent probe, periodic
|
||||
rebalancing loop, or manual layer selector. It supplies standard content intent,
|
||||
the selected sender ceiling, and readback of what the browser accepted.
|
||||
|
||||
The inspected Chromium/libwebrtc source chain makes a screen-only shortcut
|
||||
especially unsafe to assume for Screener. The JavaScript `motion` hint reaches
|
||||
`VideoRtpSender` as `kFluid`, which explicitly clears `options.is_screencast`.
|
||||
`WebRtcVideoSendStream` then configures the encoder as `kRealtimeVideo`, while
|
||||
`VideoStreamEncoder` only reinterprets `BALANCED` as `MAINTAIN_RESOLUTION` when
|
||||
its encoder is marked as screen share. The explicit `balanced` preference still
|
||||
wins the earlier preference-selection branch, but the later screen-only remap
|
||||
does not follow from this project path. Treat the effective tradeoff as an
|
||||
implementation detail to verify, not a product guarantee.
|
||||
The active topology remains availability-driven. A hard connection failure,
|
||||
non-paused 15-second decoded-frame stall, parent departure, or capacity
|
||||
invalidation may trigger the ADR-0005 recovery operation. Loss, RTT, jitter,
|
||||
bitrate, resolution, FPS, freeze counters, codec, and limitation reason are
|
||||
diagnostic and cannot prove that another parent would be better.
|
||||
|
||||
An older libwebrtc experiment could detect animated screen content under
|
||||
`balanced` and cap it to 1280x720. Commit `1d7d0e6e2c50` deleted that experiment
|
||||
on 2024-05-22; the commit says it had already been disabled for several years
|
||||
and was not maintained. Current Chromium behavior must therefore not be
|
||||
described as automatically detecting game motion and forcing a 720p cap.
|
||||
## Browser VP8 Evidence
|
||||
|
||||
## 2026-08-25 Exact-Production VP8 Gate
|
||||
Chrome 151 exact-production tests with real `getDisplayMedia()` showed that
|
||||
direct and Browser-relay VP8 can sustain about 60 fps after the stock bandwidth
|
||||
estimator warms up. The observed ramp took roughly 25 to 30 seconds; a configured
|
||||
60 fps or bitrate remains a ceiling rather than a startup promise.
|
||||
|
||||
Chrome 151 exercised exact production `bf32859` with real `getDisplayMedia()`, a
|
||||
continuously changing source, separate Browser processes, VP8/libvpx, and no
|
||||
video content hint. The direct path stabilized near 59.5 encoded, received,
|
||||
decoded, and rendered fps at 1904x928 and about 7.5 Mbps. A Browser relay kept
|
||||
every stage near 59.4--59.7 fps; its outbound adapted to 1428x696 under the
|
||||
available bandwidth without losing frame cadence. Both paths needed about
|
||||
25--30 seconds for stock bandwidth-estimation ramp-up. This closes the reported
|
||||
intrinsic VP8/no-hint localhost 60 fps shortfall; sender ceilings are not startup
|
||||
or delivery guarantees, and no application start-bitrate hack follows.
|
||||
One controlled no-hint/motion comparison showed that Chromium may preserve
|
||||
motion by spatially downscaling when `motion` is set. That is the standard
|
||||
content-hint tradeoff for games, not a minimum-resolution contract. Leaving the
|
||||
hint unset can preserve screen-classified resolution while shifting pressure to
|
||||
quantization or frame delivery, so the earlier no-hint improvement was not free
|
||||
adaptation. Current policy follows the standard game-motion intent and leaves
|
||||
the resulting tradeoff to the browser.
|
||||
|
||||
The exact SFU path isolated a different issue. With the then-deployed ordered `q,h`
|
||||
publication, the 1904x928 `h` stream stabilized near 9.2 fps and 1.37 Mbps. With
|
||||
`q` inactive from the first sender-parameter application, the otherwise
|
||||
unchanged `h` stream reached about 23.3 fps and 3.32 Mbps. Source/capture stayed
|
||||
near 60 fps, encode cost was about 4.46 ms per frame, packet loss and
|
||||
retransmission were zero, and the selected pair reported about 4.81 Mbps
|
||||
available outgoing bitrate. The always-active lower representation therefore
|
||||
consumed the same Host-to-SFU congestion budget and materially reduced `HIGH`.
|
||||
This establishes a real dual-encode cost but does not justify removing the lower
|
||||
representation needed by constrained subscribers. The public SFU ingress
|
||||
remained constrained, so this gate does not claim SFU 60 fps or real-game
|
||||
performance.
|
||||
The Host configures a sender before the first offer and reapplies the current
|
||||
video profile after accepting an answer. The latter is necessary because
|
||||
negotiation may replace or rewrite encoding parameters; it is not a periodic
|
||||
controller and does not reset bandwidth estimation on a timer.
|
||||
|
||||
## 2026-08-25 Exact-Production Single-Representation Gate
|
||||
## LiveKit SFU Evidence
|
||||
|
||||
Chrome 151 exercised exact production `1d87615` with a real dynamic tab capture.
|
||||
The bounded gate made only the ordinary STUN peer attempt unusable so the existing
|
||||
route operation reached its SFU suffix; Browser LiveKit publisher and subscriber
|
||||
PCs retained empty external ICE-server lists and selected UDP. The publisher
|
||||
reported one video sender encoding and one outbound video stats object, both
|
||||
without RID. Over six seconds it encoded 180 frames and the subscriber decoded
|
||||
and rendered 182 frames at 1822x1080, with byte counters advancing on both ends.
|
||||
This verifies the deployed single-representation baseline. It does not prove
|
||||
that the baseline serves constrained subscribers and does not close the later
|
||||
native-adaptation decision. It also does not establish public SFU 60 fps,
|
||||
real-game quality, heterogeneous networks, or endurance.
|
||||
Pinned LiveKit client `2.22.0`, server `1.13.5`, and Chrome 151 established that
|
||||
the SDK's default VP8 screen-share simulcast can expose original and lower
|
||||
representations, and that server send-side BWE can select a lower representation
|
||||
for a constrained subscriber while an unconstrained subscriber receives the
|
||||
highest available representation.
|
||||
|
||||
## 2026-08-25 Pinned Native-Adaptation Gate
|
||||
An earlier exact-production A/B also showed that forcing an always-active lower
|
||||
encoding can consume the same Host-to-SFU congestion budget and reduce the
|
||||
highest encoding. That evidence rejects Screener mutation of active layers; it
|
||||
does not justify single HIGH, which abandons constrained subscribers.
|
||||
|
||||
LiveKit server `1.13.5`, JS client `2.22.0`, and Chrome 151 exercised a changing
|
||||
VP8 screen-share publication with full- and half-resolution representations.
|
||||
Manual maximum/lower subscriber selection delivered `1280x720` and `640x360`
|
||||
at about 30 fps, and the same connection returned from the lower to the full
|
||||
representation. AdaptiveStream selected full resolution for a large attached
|
||||
element, half resolution for a small one, paused when hidden, and recovered when
|
||||
visible. Without an attached element it received only the lower representation,
|
||||
so it cannot govern a Viewer that may forward the stream to peer children.
|
||||
Current source therefore:
|
||||
|
||||
Dynacast stopped representations with no subscribers after about five seconds.
|
||||
Lower-only demand stopped the full representation, but maximum demand kept both
|
||||
encodings active. It therefore avoids unused work but cannot remove VP8's
|
||||
dual-encode cost while a healthy maximum-quality subscriber exists.
|
||||
- creates one Host publication for all SFU subscribers;
|
||||
- sets VP8, no backup codec, the selected HIGH ceiling, and degradation
|
||||
preference;
|
||||
- leaves `screenShareSimulcastLayers` unset so pinned LiveKit owns construction
|
||||
and republish behavior;
|
||||
- enables Dynacast and server send-side BWE;
|
||||
- leaves AdaptiveStream disabled because a Viewer may relay the received track
|
||||
to children; and
|
||||
- requests no application-selected subscriber layer.
|
||||
|
||||
With LiveKit's default receiver-side BWE, an approximately 0.96 Mbps subscriber
|
||||
remained on HIGH for about 21 seconds and ended at zero decoded fps. With
|
||||
`congestion_control.use_send_side_bwe: true`, LiveKit's native stream allocator
|
||||
selected LOW and by the third steady window delivered `640x360` at about 29 fps
|
||||
and 88 KB/s. A fresh unshaped subscriber immediately received HIGH. Network
|
||||
unshaping on the same PeerConnection was not isolated because the Chrome DevTools
|
||||
condition was bound when that connection was created; manual and AdaptiveStream
|
||||
same-connection upward transitions independently proved layer recovery.
|
||||
LiveKit documents automatic bandwidth-based layer selection. It does not expose
|
||||
an automatic policy that observes delivered FPS and spatially downshifts solely
|
||||
to satisfy Screener's `maintain-framerate` preference. Adding such a controller
|
||||
or fork remains outside the current product.
|
||||
|
||||
The result accepts pinned-default VP8 dual-representation simulcast, Dynacast,
|
||||
and server send-side BWE.
|
||||
Screener does not add a bandwidth estimator or layer controller. AdaptiveStream
|
||||
stays disabled for current subscribers because every Viewer may become a relay.
|
||||
The earlier dual-encode cost is handled through explicit Host share profiles,
|
||||
not by silently abandoning low-bandwidth Viewers.
|
||||
## Codec Decision
|
||||
|
||||
## 2026-08-25 Content Classification And Bounded Scaling Gate
|
||||
VP8 is the only Browser media codec because it has the broadest current Browser
|
||||
interoperability and produced stable software-encoding behavior in the measured
|
||||
Chrome path. The page does not select a particular VP8 hardware or software
|
||||
encoder.
|
||||
|
||||
Chrome `151.0.7922.174` exercised one real `getDisplayMedia()` track, one VP8
|
||||
P2P sender/receiver pair, `maintain-framerate`, and the same deterministic
|
||||
moving 1904x928 source for 30-second windows. At an 8 Mbps ceiling, no video
|
||||
hint held about 60 fps at 1904x928 and about 8 Mbps with no resolution changes.
|
||||
`contentHint = "motion"` held 60 fps but the native quality scaler reduced the
|
||||
track and sender to 242x118 and remained there through the steady window.
|
||||
Applying an 854x480 minimum track constraint before the motion run did not form
|
||||
an encoder floor: the steady sender was 428x208 at about 60 fps and 8 Mbps.
|
||||
Controlled Windows Chrome evidence localized the poor H.264 result to Chromium's
|
||||
rate-control path around some Media Foundation encoders rather than a portable
|
||||
page setting. The same AMD MFT reached the requested cadence only under a
|
||||
diagnostic browser field trial that a Web application cannot enable. OpenH264
|
||||
software was not a superior fallback. H.264 remains research evidence for a
|
||||
later native sender, not a Browser option.
|
||||
|
||||
The result matches Chrome's exact libwebrtc pin: VP8 automatic resize is disabled
|
||||
for a screencast and enabled only for a realtime, single-active-stream encoder.
|
||||
The motion hint therefore restores native bandwidth/QP spatial scaling by
|
||||
changing the whole content classification, not by enabling an independent
|
||||
quality switch. The available standard track constraint does not bound that
|
||||
internal encoder scaling. Restoring motion would recover multiple resolution
|
||||
steps but can make game UI unreadable, so current source must not restore it as
|
||||
a global or fluid-profile shortcut.
|
||||
HEVC, AV1, custom WebCodecs pipelines, and application packetization do not
|
||||
replace the browser WebRTC sender without a new capture, RTP/RTCP, feedback,
|
||||
hardware, interoperability, and licensing design. No such path is accepted.
|
||||
|
||||
This gate proves the mechanism, not visual quality for every game. It also means
|
||||
the no-hint result is not a free efficiency gain: it preserves screen encoding
|
||||
semantics and resolution while shifting pressure into quantization or frame
|
||||
drops. A future bounded P2P spatial controller is a separate product mechanism,
|
||||
not a hidden consequence of `degradationPreference`.
|
||||
## Host Cost Boundary
|
||||
|
||||
The same production session exposed the separate SFU ingress boundary. A
|
||||
1440p60/8 Mbps publication advertised a same-FPS half-resolution 2 Mbps lower
|
||||
representation. The SFU Viewer received the highest representation at
|
||||
2400x1350, about 37 fps and 1.84 Mbps while its downlink reported 54 ms RTT and
|
||||
zero loss; the simultaneous P2P Viewer received about 59 fps and 5 Mbps. A
|
||||
server interval measured about 4.46 Mbps ingress and 2.63 Mbps egress, consistent
|
||||
with the unforwarded lower representation consuming Host-to-SFU budget. Later,
|
||||
the native allocator selected only LOW and an RTP-layer sample observed LOW
|
||||
traffic with no HIGH traffic, so downstream layer selection itself was not
|
||||
stuck. The accepted correction keeps that native selector and reduces only the
|
||||
60 fps lower representation to at most 30 fps using the SDK's proportional
|
||||
bitrate formula.
|
||||
One Chrome 151 VP8 sample on a Ryzen 7 9700X used about 0.79 logical core for one
|
||||
1904x928@30 sender and about 1.34 logical cores for two independent senders. The
|
||||
process totals included capture, WebRTC, source rendering, and local decode, so
|
||||
they are not a weak-device or real-game capacity claim. Browser peers retain one
|
||||
independent sender and congestion controller per `RTCPeerConnection`; shared
|
||||
encoding is not guaranteed.
|
||||
|
||||
The accepted way to handle a constrained Host is the existing explicit share
|
||||
profile. Screener does not silently remove a representation needed by a weak
|
||||
Viewer or invent a hidden host-performance tier.
|
||||
|
||||
## Page Lifecycle Boundary
|
||||
|
||||
Host document visibility, captured-source visibility, capture production,
|
||||
encoder throughput, transport, Viewer decode, and local presentation are
|
||||
different variables. Current code pauses only the Host's muted local preview
|
||||
when its page is hidden or unfocused; it does not stop capture or senders.
|
||||
|
||||
Browser timers and frame callbacks may be delayed while hidden. A resumed Viewer
|
||||
must rebaseline the existing decoded-stall wall clock before another no-progress
|
||||
sample can invalidate the route, and a new current-generation composited frame
|
||||
is authoritative evidence that stale `connecting` or `reconnecting`
|
||||
presentation has ended. These rules make recovery lifecycle-safe; they are not
|
||||
a keepalive mechanism.
|
||||
|
||||
The Screen Capture specification permits a user agent to mute a captured surface
|
||||
that becomes inaccessible, including a minimized captured window. A page cannot
|
||||
override that platform boundary. Mobile background audio, video recovery, page
|
||||
reclamation, and relay survival remain physical-device acceptance work.
|
||||
|
||||
## Route Quality Authority
|
||||
|
||||
Current-path WebRTC stats can diagnose loss, RTT, jitter, bitrate, resolution,
|
||||
FPS, freeze, encode/decode work, and limitation reason, but they cannot prove the
|
||||
counterfactual quality of an unconnected parent. RFC 8836 assigns real-time
|
||||
congestion adaptation to the media transport. The Overcast, Narada, and NICE
|
||||
overlay algorithms require active measurements, periodic reevaluation,
|
||||
thresholds, or additional overlay state; those mechanisms conflict with
|
||||
Screener's one event-driven reconciliation loop, one room-serial child
|
||||
operation, and sticky healthy edges.
|
||||
|
||||
Screener therefore does not compute a weighted route score, probe alternative
|
||||
parents, or reparent a currently decoding edge for bitrate, resolution, FPS,
|
||||
freeze ratio, RTT, jitter, loss, or limitation evidence. Only a hard
|
||||
`failed/closed` connection or the existing non-paused decoded-frame stall makes
|
||||
the exact edge invalid. The current wire carries no parent-wide quality evidence
|
||||
or threshold-based route authority. Primary sources checked 2026-08-24: [RFC
|
||||
8836](https://www.rfc-editor.org/rfc/rfc8836.html),
|
||||
[Overcast](https://www.usenix.org/legacy/publications/library/proceedings/osdi2000/full_papers/jannotti/jannotti_html/index.html),
|
||||
[Narada](https://www.cs.cmu.edu/~srini/papers/papers/2002-Chu-jsac/2002-Chu-jsac.pdf),
|
||||
and [NICE](https://conferences.sigcomm.org/sigcomm/2002/papers/appmulti.pdf).
|
||||
|
||||
## 2026-08-19 Then-Production Degradation Report
|
||||
|
||||
The then-production release `769de201f7cc` was reported to become severely blurry
|
||||
while the host repeatedly showed a sustained native
|
||||
`qualityLimitationReason=bandwidth`, despite one capable viewer on the same LAN
|
||||
and the UI showing direct P2P. This is a user report, not yet an instrumented
|
||||
root cause. The probe is read-only, requires three consecutive windows, and
|
||||
does not change media. Same-LAN/direct does not exclude Wi-Fi loss or queuing,
|
||||
per-PeerConnection GCC, aggregate host-uplink contention, or browser bandwidth
|
||||
underestimation; isolated encoder pressure would more commonly report `cpu`.
|
||||
|
||||
Only a full Host refresh and re-share was reported to restore immediate
|
||||
high-definition, fluid video; Viewer refresh did not. With a stable
|
||||
session-storage `clientId` and reconnect inside the five-second grace, Viewer
|
||||
reauth reuses its `peerId` and emits `peer-joined`, while `HostPage.startPeer`
|
||||
retains an already-connected `HostPeer`; a new tab, storage failure, or expired
|
||||
grace can instead allocate a new peer. The reported Viewer refresh therefore
|
||||
did not create a new host PeerConnection/GCC generation. Host-page cleanup
|
||||
disposes every peer,
|
||||
stops capture, and creates all-new capture, sender, PC, and GCC generations.
|
||||
This makes a stuck host sender/PC/GCC or shared-capture generation a priority
|
||||
hypothesis, not a conclusion. A further report that quality may remain low
|
||||
after the old viewer leaves and a new viewer joins must likewise be tested
|
||||
against lifecycle evidence rather than assumed.
|
||||
|
||||
Use this as the first A+B/C reproduction before any route change or dual-layer
|
||||
publication:
|
||||
|
||||
1. Reproduce one wired LAN/direct viewer first, then Wi-Fi, two/three viewers,
|
||||
and SFU/UDP, using the same high-motion scene for 60 to 90 seconds.
|
||||
2. In one window record A capture; B actual/target/available outbound bitrate,
|
||||
dimensions/FPS, codec, encode cost, loss/RTT/retransmission, selected path,
|
||||
limitation, and PC/track generations; and C inbound decode/drop/freeze.
|
||||
3. First retain capture and every healthy peer while rebuilding only the
|
||||
affected `HostPeer`/`connectionId`. Separately compare capture/track
|
||||
replacement with PCs retained, then a full Host rebuild; do not combine the
|
||||
interventions.
|
||||
4. For viewer churn and stable-`clientId` reconnect, retain old/new
|
||||
`peerId`/`connectionId`, `peer-left`-to-dispose time, active sender count
|
||||
inside/outside the five-second grace, old snapshot removal, whether a new PC
|
||||
actually exists, capture settings, and new-path BWE/limitation.
|
||||
|
||||
A single-edge rebuild passes only if the affected viewer recovers, unaffected
|
||||
viewers neither migrate nor interrupt, and the host-edge cap holds. Only proven
|
||||
generation-specific hard failure or non-paused decoded-frame stall can justify
|
||||
automatic one-edge recovery under the current route attempt. Do not add
|
||||
periodic reconnect, blindly raise a ceiling, or
|
||||
change the current routing behavior before this gate.
|
||||
|
||||
Missing `scaleResolutionDownBy` is not itself a root cause for a single encoding
|
||||
because its effective default is 1.0. Likewise, the current 3/5/8 Mbps values
|
||||
are ceilings rather than targets. Raising them cannot repair CPU or bandwidth
|
||||
limitation and should only follow evidence that the encoder is already pinned
|
||||
to the ceiling while spare transport capacity remains.
|
||||
|
||||
## Confounded Same-Setting Observation
|
||||
|
||||
On 2026-08-21 one bounded local Chrome 151 run used exact source `b447ab6`, a
|
||||
synthetic 1920x1080 at 60 fps capture, and one direct UDP Viewer. After the
|
||||
initial connection had existed for about 22 seconds, the Host outbound and
|
||||
Viewer inbound were both 1280x720 at 57 fps and about 7.03 Mbps. The sender
|
||||
reported `qualityLimitationReason=bandwidth`, despite about 12.59 Mbps of
|
||||
available outgoing bitrate. Reapplying the identical 1080p60 `balanced` setting
|
||||
through the real Host control produced 1920x1080 at 55 fps and about 7.11 Mbps
|
||||
after 5.2 seconds, with the limitation cleared. Both ends reported zero packet
|
||||
loss and no Viewer freeze.
|
||||
|
||||
The route revision, PeerConnection identity, RTP SSRC, RTP stats object, and
|
||||
track identifier all remained unchanged. This rules out a new route, sender,
|
||||
or capture generation as the source of that observed transition. It does not
|
||||
prove visual quality, production behavior, or SFU behavior. Earlier samples in
|
||||
the same run remained at 640x360 through roughly 20 seconds and reached 720p in
|
||||
the two-second window immediately before the control action, so continued
|
||||
stock BWE ramp-up remains a causal confound. No second browser run was made.
|
||||
The Host control applies same-track capture constraints before updating sender
|
||||
parameters, so this run also cannot isolate those two standard calls.
|
||||
|
||||
Static review does not support the simpler claim that Chromium discards every
|
||||
pre-negotiation `setParameters()` call. Current libwebrtc stores sender init
|
||||
parameters while no SSRC exists and transfers them into the negotiated sender.
|
||||
The observed transition followed the same-setting action, but this single run
|
||||
cannot attribute it to sender lifecycle or the reapply itself and therefore
|
||||
does not support an answer-time whole-profile workaround. Current source keeps
|
||||
the pre-offer sender configuration and serialized explicit source/profile
|
||||
updates, but an accepted answer itself does not rewrite the profile. A future
|
||||
workaround requires a controlled, real-capture reproduction that isolates one
|
||||
intervention from stock BWE ramp-up.
|
||||
|
||||
This observation alone did not justify an SFU change because stock BWE ramp-up
|
||||
was not isolated. The later exact-production gate above independently measured
|
||||
the SFU publication and rejected its always-active lower representation.
|
||||
LiveKit's pinned client contains an SDP start-bitrate mitigation for initial
|
||||
video blur, but Screener does not add application SDP bitrate hacks.
|
||||
|
||||
The same release was also reported to reduce game-stream frame rate and
|
||||
consume noticeable Host resources. That report applies only to
|
||||
`769de201f7cc`, not to current production or automatically to the current
|
||||
diagnostics.
|
||||
The Web
|
||||
sender creates one independent `RTCRtpSender` per viewer and has no cross-PC
|
||||
shared-encoder guarantee; its muted local preview creates no media edge or
|
||||
server traffic but may still consume compositor/GPU work. Compare the exact
|
||||
release and exact current source under one fixture, with preview on/off as a separate
|
||||
binary intervention.
|
||||
|
||||
Desktop background diagnosis must also separate three independent variables:
|
||||
Host document foreground/occluded/minimized state, captured source
|
||||
foreground/occluded/minimized state, and local preview play/pause. Record
|
||||
document visibility/focus, capture-track `muted`/`readyState`, actual codec,
|
||||
capture/outbound/inbound FPS and bitrate, resolution, frames, encode time, and
|
||||
limitation reason. The Screen Capture specification does not expose a page
|
||||
keepalive contract; Screen Wake Lock is released when its document is no longer
|
||||
visible, and browser page lifecycle may freeze JavaScript. Do not add fake
|
||||
activity or keepalive timers. If the capture source or browser/OS suppresses
|
||||
frames while minimized, record that standard capability boundary before any
|
||||
packaged Host work. Primary sources checked 2026-08-24: [Screen
|
||||
Capture](https://www.w3.org/TR/screen-capture/), [Screen Wake
|
||||
Lock](https://www.w3.org/TR/screen-wake-lock/), [Page Lifecycle
|
||||
API](https://developer.chrome.com/docs/web-platform/page-lifecycle-api), and
|
||||
[Chromium desktop capture
|
||||
implementation](https://chromium.googlesource.com/chromium/src/+/master/content/browser/media/capture/desktop_capture_device.cc).
|
||||
|
||||
## SFU Profile Lifecycle And Publisher Evidence
|
||||
|
||||
A 2026-08-21 report says that selecting fluid preference on an SFU path could
|
||||
retain low received FPS without reducing the visible resolution. This is not
|
||||
proof that the preference was ignored: `maintain-framerate` is a degradation
|
||||
tradeoff rather than an FPS target. It is also not evidence of SFU temporal
|
||||
downlayering: the then-deployed v10 Browser defaulted to VP8 but let the Host
|
||||
select another value before starting the share. Pinned LiveKit server 1.13.5 installs a temporal selector
|
||||
for VP8, but its H.264/H.265 path installs only the simulcast spatial selector.
|
||||
With H.264 selected, a `HIGH` ceiling may therefore let BWE choose the
|
||||
lower-resolution `q` representation, not a lower temporal layer at the same
|
||||
resolution. When resolution remains stable, classify the case in A/B/C order:
|
||||
capture FPS, Host publisher outbound FPS and limitation, then Viewer inbound FPS
|
||||
and decode.
|
||||
|
||||
Pinned LiveKit client 2.22.0 keeps three relevant pieces of state. Its public
|
||||
`LocalVideoTrack.setDegradationPreference()` updates the saved preference used
|
||||
when a sender is installed; `LocalVideoTrack.publishOptions` drives encoding
|
||||
recomputation after a track restart; and `LocalTrackPublication.options` is the
|
||||
input to `republishAllTracks()`. Current source keeps those three owners aligned
|
||||
through the existing publisher operation queue and rollback: call the SDK
|
||||
preference API to update its saved state, configure/read back the exact single
|
||||
sender as the final write, then assign one merged option object to both retained
|
||||
locations. Failure reapplies the previous profile; generation loss cannot
|
||||
retain the result as current publisher state. This changes no capture
|
||||
constraint, codec, representation, subscriber layer or route policy and does
|
||||
not explain an immediate same-publication report.
|
||||
|
||||
Host SFU publisher A+B remains a separate observability slice. Current source
|
||||
implements one two-second, publication-generation-bound local sampler owned by
|
||||
`SfuPublisher`: merge its video/audio `LocalTrack` reports, reuse the existing
|
||||
strict stats parser and accumulator, correlate capture settings from the owned
|
||||
video track, and emit only while the same publication is active. Activation,
|
||||
replacement, deactivation and disconnect reset its identity and interval
|
||||
baseline. `HostSfuRoute` may expose that local snapshot to one Host-only
|
||||
publisher row; it must not duplicate the shared Host-to-SFU ingress inside each
|
||||
SFU Viewer card. Viewer inbound remains the per-Viewer C signal. This needs no
|
||||
wire, server telemetry, global score, selector or new UI framework. Production
|
||||
deploys this publisher view; its target-browser fields and values still need
|
||||
physical evidence.
|
||||
|
||||
## Browser Codec And Content-Hint Evidence
|
||||
|
||||
The accepted Browser contract is fixed VP8 with no video `contentHint`; returned
|
||||
audio keeps `contentHint = "music"`. Direct and browser-relay offers contain VP8
|
||||
as their only video media codec, while RTX/RED/FEC may remain repair formats;
|
||||
SFU publication explicitly uses VP8 with no backup codec. The UI and quality
|
||||
wire expose no codec choice. Codec/profile/encoder stats remain diagnostic and
|
||||
do not authorize automatic switching, route changes, or another controller.
|
||||
Production runs exact deployed application/runtime revision
|
||||
`be53c4d5794d38d5d406c876309958d7117ee601`, release `be53c4d`. Current source and production implement
|
||||
this strict `screener-v12` contract.
|
||||
|
||||
Chromium maps video `contentHint = "motion"` to libwebrtc `kFluid`, and
|
||||
libwebrtc clears `is_screencast` for that mode, replacing the display-capture
|
||||
screen classification with realtime-camera semantics. Fresh
|
||||
Chrome 151 loopback probes on 2026-08-24 used real dynamic tab
|
||||
`getDisplayMedia()`, `balanced`, ten seconds of warm-up, and a fifteen-second
|
||||
sample:
|
||||
|
||||
| Requested path | Ceiling | Actual encoder | Encoded FPS | Encode time/frame | Bitrate | End resolution | Limitation |
|
||||
| --- | ---: | --- | ---: | ---: | ---: | --- | --- |
|
||||
| VP8 + `motion` | 5 Mbps | libvpx software | 14.93 | 2.85 ms | 3.663 Mbps | 428x208 | `bandwidth` |
|
||||
| VP8 + no hint | 5 Mbps | libvpx software | 29.73 | 6.62 ms | 4.944 Mbps | 1904x928 | `none` |
|
||||
| H.264 + no hint | 8 Mbps | AMD Media Foundation | 14.18 | 14.24 ms | 8.214 Mbps | 1904x928 | `none` |
|
||||
| H.264 + no hint + diagnostic Desktop SW BRC | 5 Mbps | AMD Media Foundation | 29.93 | 8.17 ms | 3.650 Mbps | 1904x928 | `none` |
|
||||
| H.264 + no hint + hardware encode disabled | 5 Mbps | OpenH264 software | 10.47 | 13.08 ms | 4.832 Mbps | 1904x928 | `none` |
|
||||
|
||||
The VP8 result establishes that `motion` was harmful independently of the H.264
|
||||
issue. VP8 reported trusted rate control, so libwebrtc's outer frame dropper was
|
||||
off and libvpx owned its CBR behavior. In the ordinary AMD H.264 path,
|
||||
libwebrtc's outer rate limiter dropped hundreds of frames; raising the ceiling
|
||||
through 8/10/12 Mbps or selecting `maintain-framerate` did not restore 30 fps.
|
||||
The diagnostic SW-BRC run made the same AMD MFT report trusted rate control and
|
||||
reach 29.93 fps, so the MFT itself is not a fixed 12--15 fps throughput limit.
|
||||
That browser-process experiment bypasses a Chromium AMD workaround and is not a
|
||||
Web product API; a page cannot enable the field trial or select a GPU/MFT.
|
||||
|
||||
A separate no-hint VP8 CPU sample used a Ryzen 7 9700X, Chrome 151, one dynamic
|
||||
1904x928@30 capture, 5 Mbps per sender, and loopback receivers. One sender held
|
||||
29.86 fps and the isolated Chrome process tree consumed 21.14 CPU-seconds over
|
||||
26.65 seconds, or 0.793 logical core / 4.96% of the 16-thread machine. Two
|
||||
independent senders both held 29.93 fps and used 35.88 CPU-seconds over 26.70
|
||||
seconds, or 1.344 logical cores / 8.40%. These totals include source rendering,
|
||||
capture, WebRTC, and local decode, so they are not pure encoder cost. They show
|
||||
that software VP8 is workable on this machine, not that it is cheap on a weaker
|
||||
CPU, mobile device, or beside a real game.
|
||||
|
||||
Chromium's generic WebRTC factory can wrap hardware VP8 when a platform backend
|
||||
advertises it, but this Windows Chrome run selected libvpx and Chromium 151's
|
||||
Windows Media Foundation backend advertises VP9, H.264, AV1, and conditional
|
||||
HEVC rather than VP8. A website can prefer a codec/profile but cannot require a
|
||||
particular hardware implementation. Hardware VP9 or AV1 may provide usable rate
|
||||
control on supported machines, while H.264/HEVC remain important hardware paths,
|
||||
but none currently combines VP8's WebRTC baseline coverage with deterministic
|
||||
hardware selection across Screener's browser targets. Capability advertisement
|
||||
alone is not acceptance; an actual sender must prove codec, implementation,
|
||||
power efficiency, rate control, and game-load behavior.
|
||||
|
||||
These results select VP8 for the ordinary Browser path: no-hint VP8 met the
|
||||
current controlled quality baseline, has the required WebRTC interoperability,
|
||||
and does not depend on a page choosing a specific Windows encoder. Exact current
|
||||
source uses `RTCRtpTransceiver.setCodecPreferences()` before
|
||||
the first offer and fails the edge if the API, a VP8 capability, or the
|
||||
preference application is missing; it does not leave another video media codec
|
||||
as fallback. Browser VP8
|
||||
hardware acceleration is a separate later evidence question. It must be proven
|
||||
on the supported browser/platform matrix from the actual encoder and game-load
|
||||
measurements before it can change the implementation; custom codecs, another
|
||||
codec ladder, and a Native helper are not implied by this decision.
|
||||
|
||||
Comparable native products do not resolve this browser boundary. Discord
|
||||
negotiates VP8/H.264 and selected-platform HEVC/AV1 through its own capture and
|
||||
hardware pipeline, and documents an AMD rate-control/frame-dropper repair.
|
||||
Parsec, Steam Remote Play, and Moonlight/Sunshine primarily use controlled
|
||||
hardware H.264/HEVC/AV1 paths. A read-only inspection of the installed Oopz
|
||||
0.87.425 package found Agora screen sharing, an H.264 Web-viewer configuration,
|
||||
and native NVENC/QSV/AMF plus software codec paths; it did not prove the codec or
|
||||
encoder active in a real sender session. These native designs do not prove that
|
||||
an ordinary web page can force the same encoder path. Reproducibility hashes and
|
||||
the exact static boundary are retained in
|
||||
[Native shared encode](./native-shared-encode-sender.md#2026-08-20-windows-codec-comparison).
|
||||
|
||||
Primary implementation evidence: Chromium's
|
||||
[content-hint bridge](https://chromium.googlesource.com/chromium/src/+/refs/tags/151.0.7922.174/third_party/blink/renderer/modules/peerconnection/media_stream_video_webrtc_sink.cc#36),
|
||||
libwebrtc's [sender classification](https://webrtc.googlesource.com/src/+/f20ebb8adbf4fa781830e4384c61f732bd28a217/pc/rtp_sender.cc#1419),
|
||||
[VP8 rate control](https://webrtc.googlesource.com/src/+/f20ebb8adbf4fa781830e4384c61f732bd28a217/modules/video_coding/codecs/vp8/libvpx_vp8_encoder.cc#1360),
|
||||
Chromium's [hardware codec mapping](https://chromium.googlesource.com/chromium/src/+/refs/tags/151.0.7922.174/third_party/blink/renderer/platform/peerconnection/rtc_video_encoder_factory.cc#75),
|
||||
its [Windows encoder backend](https://chromium.googlesource.com/chromium/src/+/refs/tags/151.0.7922.174/media/gpu/windows/media_foundation_video_encode_accelerator_win.cc#606),
|
||||
the [Desktop SW BRC switches](https://chromium.googlesource.com/chromium/src/+/refs/tags/151.0.7922.174/media/gpu/windows/mf_video_encoder_switches.cc#26),
|
||||
and Chromium's [AMD workaround](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/gpu/config/gpu_driver_bug_list.json#3303).
|
||||
Product comparisons: [Discord Go Live](https://discord.com/blog/how-it-all-goes-live-an-overview-of-discords-streaming-technology),
|
||||
[Discord AMD rate control](https://discord.com/blog/from-blocky-to-brilliant-improving-video-quality-on-discord-go-live-on-amd-gpus),
|
||||
[Oopz privacy policy](https://help.oopz.cn/agreement/privacy), and
|
||||
[Agora codec selection](https://doc.shengwang.cn/api-ref/rtc/windows/API/enum_videocodectype),
|
||||
[Parsec compatibility](https://support.parsec.app/hc/en-us/articles/32381568346644-Hardware-and-Software-Compatibility),
|
||||
[Steam Remote Play update](https://store.steampowered.com/news/posts/?enddate=1734653759&feed=steam_clientAny),
|
||||
[Moonlight](https://github.com/moonlight-stream/moonlight-qt), and
|
||||
[Sunshine](https://github.com/LizardByte/Sunshine/blob/master/docs/configuration.md).
|
||||
|
||||
## Evidence Before Adaptation
|
||||
|
||||
Verified specification facts: the W3C stats model supports the needed
|
||||
separation but does not produce the product decision itself.
|
||||
`MediaStreamTrack.getSettings()` supplies the track's current configured
|
||||
dimensions/frame rate, not the recent frame cadence. `RTCVideoSourceStats`
|
||||
supplies the last-second FPS actually fed to the encoder. Outbound RTP exposes
|
||||
emitted dimensions/FPS, byte and retransmission counters, encode time, target
|
||||
bitrate when available, and the current `qualityLimitationReason`; the selected
|
||||
candidate pair can expose RTT and available outgoing bitrate. Inbound RTP
|
||||
exposes received dimensions/FPS, byte/loss/jitter counters, decoded/dropped
|
||||
frames, and freeze counters when implemented. Stats members may be absent, and
|
||||
cumulative values must be compared across two samples rather than treated as
|
||||
interval values.
|
||||
|
||||
The user-facing loss value should therefore be an RTP interval loss rate, not
|
||||
the cumulative `packetsLost` counter and not a claim about UDP itself. For one
|
||||
unchanged RTP stats object, compute `lostDelta / (receivedDelta + lostDelta)`;
|
||||
an initial sample, reset, negative delta, zero denominator, or absent field is
|
||||
unknown. `RTCReceivedRtpStreamStats` defines both counters and notes that
|
||||
`packetsLost` can be negative. `RTCIceCandidateStats.address` is privacy-sensitive
|
||||
and remote addresses are null by default unless the application supplied the
|
||||
candidate. Screener may show a selected address locally in collapsed details,
|
||||
but it must never enter Viewer C, signaling, logs, exports, persistence, route
|
||||
selection, or NAT/geography inference.
|
||||
|
||||
Correlate one time interval and media generation across:
|
||||
|
||||
| Evidence | Fields |
|
||||
| --- | --- |
|
||||
| A. Host capture | configured width, height, FPS from `getSettings()` plus observed `media-source` input FPS |
|
||||
| B. Host outbound | RID, width/FPS/bitrate, target/available bitrate, interval encoded frames/encode time, limitation reason, path, RTT, loss/retransmission, and derived negotiated codec/profile/parameters plus applicable `scalabilityMode` |
|
||||
| C. Viewer inbound | width/FPS/bitrate, loss, jitter, interval decode/drop/freeze, corresponding derived codec/profile/parameters/layer, and actual decode behavior |
|
||||
|
||||
Product inference from those facts: use the following ordered classification:
|
||||
|
||||
| Correlated observation | Likely boundary |
|
||||
| --- | --- |
|
||||
| Capture low | capture or constraints |
|
||||
| Capture high; outbound low; `cpu` | encoder/resource pressure |
|
||||
| Capture high; outbound low; `bandwidth` | GCC, uplink, or current-path pressure |
|
||||
| Outbound healthy; inbound low | transport or receiver path |
|
||||
| Inbound healthy; image visibly blurry | bitrate/quantization, codec, or display scaling |
|
||||
|
||||
This is a diagnostic classification, not a weighted health score. Missing or
|
||||
reset evidence remains unknown and rebases the interval.
|
||||
|
||||
The smallest implementation sequence is local A+B correlation in one host
|
||||
sampling tick, followed by a minimal authenticated C report carrying the
|
||||
receive/decode and derived negotiation signals needed by the topology
|
||||
predicate. It never carries raw SDP, raw stats, candidate addresses, or raw
|
||||
device/network identifiers. Server-authoritative path and connection generations
|
||||
provide authorization and correlation; a general remote stats stream or
|
||||
telemetry pipeline is unnecessary.
|
||||
|
||||
The repository implements the local host A+B foundation: same-tick track
|
||||
settings, linked `media-source` input FPS, and one uniquely matched outbound RTP
|
||||
sample with its RID, explicit sample/media identity and adjacent frame/encode
|
||||
time deltas, `remoteId` linkage, and the selected path reached through that RTP
|
||||
stream's transport. Source replacement blocks sampling and invalidates in-flight
|
||||
generations. Direct/relay edges bind that evidence to the
|
||||
current track and PeerConnection. The Host SFU publisher also samples only its
|
||||
exact active LiveKit sender and binds the result again to the
|
||||
current publication generation; replacement, profile reset, retirement,
|
||||
disconnect, and authoritative resync clear the old identity before another
|
||||
sample can appear. The Host details UI renders that source once as `SFU 发送`,
|
||||
not once per SFU-fed Viewer. This is local diagnostics only: it creates no wire,
|
||||
score, selector, or media action.
|
||||
|
||||
Authenticated Viewer C is also implemented for each current ordinary or
|
||||
peer-assisted P2P hop. A viewer sends one nullable, sanitized aggregate window
|
||||
on the existing two-second stats cadence, bounded to 2 KiB; the server derives
|
||||
the current viewer, parent, connection, and route revision, while the parent
|
||||
accepts only the matching local generation and expires it. The report contains
|
||||
no raw SDP, stats, candidates, addresses, device identifiers, or room identity,
|
||||
is neither stored nor used for media or routing action, and fails closed for an
|
||||
SFU-fed root until a real SFU last-hop B/generation exists.
|
||||
|
||||
Codec evidence follows only that outbound RTP object's `codecId`, and the
|
||||
referenced `RTCCodecStats` must use the same transport. The local diagnostics
|
||||
expose nullable, bounded MIME plus a normalized profile token and an ASCII allowlist of
|
||||
format parameters: H.264 `profile-level-id`, `packetization-mode`, and
|
||||
`level-asymmetry-allowed`; VP9 `profile-id`, `max-fr`, and `max-fs`; VP8
|
||||
`max-fr` and `max-fs`; and AV1 `profile`, `level-idx`, and `tier`. Unknown,
|
||||
invalid, duplicated, oversized, or non-allowlisted fields such as H.264
|
||||
parameter sets are discarded. This diagnostic path does not expose or persist
|
||||
raw `sdpFmtpLine`, SDP, candidates, or complete stats, and it does not infer
|
||||
specification defaults. Profile tokens remain self-describing, for example
|
||||
H.264 `profile-level-id=42e01f`; they are not translated into quality,
|
||||
hardware, support, or capability conclusions.
|
||||
|
||||
The same unique outbound object supplies nullable current configured
|
||||
`scalabilityMode`. Sender parameter readback exposes an applied mode only for
|
||||
one unambiguous encoding. Current quality settings do not request a mode, so
|
||||
the requested value remains null and a browser-reported default is not called
|
||||
a mismatch; multiple encodings remain unknown. Inbound stats provide no current
|
||||
standard `scalabilityMode` source, so C does not carry a null-only placeholder.
|
||||
The earlier exact-production gate rejected its ordered `q,h` publication because
|
||||
the active lower representation reduced `HIGH`. Current source implements the
|
||||
later accepted LiveKit-owned two-representation target; production remains the
|
||||
exact no-RID single-`HIGH` baseline until cutover.
|
||||
|
||||
Official W3C text checked 2026-08-19 defines names ending in `Id` as stats-object
|
||||
references. In particular, outbound [`mediaSourceId`](https://www.w3.org/TR/webrtc-stats/#dom-rtcoutboundrtpstreamstats-mediasourceid)
|
||||
references the sender's current media source, whose
|
||||
[`trackIdentifier`](https://www.w3.org/TR/webrtc-stats/#dom-rtcmediasourcestats-trackidentifier)
|
||||
is the `MediaStreamTrack.id`; RTP [`transportId`](https://www.w3.org/TR/webrtc-stats/#dom-rtcrtpstreamstats-transportid)
|
||||
references its transport, and that transport's
|
||||
[`selectedCandidatePairId`](https://www.w3.org/TR/webrtc-stats/#dom-rtctransportstats-selectedcandidatepairid)
|
||||
references its selected pair. Therefore an ambiguous RTP set or a broken or
|
||||
cross-transport reference remains unknown; an opaque stats ID is not a safe
|
||||
substitute for those relationships.
|
||||
|
||||
The same specification defines [`codecId`](https://www.w3.org/TR/webrtc-stats/#dom-rtcrtpstreamstats-codecid)
|
||||
as the RTP stream's reference to `RTCCodecStats`, whose `sdpFmtpLine` contains
|
||||
format-specific negotiated parameters, and defines outbound
|
||||
[`scalabilityMode`](https://www.w3.org/TR/webrtc-stats/#dom-rtcoutboundrtpstreamstats-scalabilitymode)
|
||||
as present only when a mode is currently configured for that stream.
|
||||
WebRTC-SVC separately permits post-negotiation `getParameters()` to return a
|
||||
different configured mode from one explicitly requested. If no mode was
|
||||
provided, its absence must not be replaced with an implementation-dependent
|
||||
default. Chromium currently maps its configured per-stream mode into the
|
||||
outbound stats field; other browser implementations may omit any optional
|
||||
member, which remains null rather than a capability conclusion.
|
||||
|
||||
## LiveKit Native Representation Boundary
|
||||
|
||||
ADR-0007 owns the accepted target: the pinned client's default VP8 screen-share
|
||||
simulcast pair with representation activation and per-subscriber selection
|
||||
delegated to LiveKit. The pinned gate passed with explicit server send-side BWE.
|
||||
Current source implements the target; production remains the not-yet-migrated
|
||||
single-encoding baseline.
|
||||
|
||||
The candidate assumed one shared dual-representation SFU publication could
|
||||
preserve full resolution while LiveKit selected a lower representation per
|
||||
constrained subscriber. Static review established two limits before the runtime gate:
|
||||
ordinary WebRTC receivers have no portable per-RID selection API across
|
||||
separate PeerConnections. Runtime evidence confirmed that LiveKit 1.13.5
|
||||
Dynacast cumulatively enables qualities below the highest requested quality, so
|
||||
a `HIGH` subscriber keeps `LOW` active. The earlier exact-production A/B proved
|
||||
that this has a material cost; the later constrained-subscriber gate proved that
|
||||
the lower representation is necessary and that LiveKit's send-side BWE selects
|
||||
it without a Screener controller. Sender/viewer evidence remains diagnostic and
|
||||
does not change topology or command a representation.
|
||||
|
||||
### Historical SFU Source Gate
|
||||
|
||||
The 2026-08-22 source-only gate verified ordered `q,h` configuration and existing
|
||||
SFU route/admission accounting, but did not start LiveKit or a Browser. The
|
||||
2026-08-25 exact-production gate supersedes that candidate's quality verdict;
|
||||
Git history retains the abandoned shaping plan.
|
||||
|
||||
SVC is the third static no-go,
|
||||
`no-go-web-svc-cross-path-hardware-contract`. WebRTC-SVC configures an outgoing
|
||||
`RTCRtpSender`; `RTCRtpReceiver` has no matching `setParameters()` experiment or
|
||||
standard base/enhancement selector. That lets an SFM/SFU selectively forward
|
||||
layers, but it does not let separate direct/peer PeerConnections consume one
|
||||
portable shared encoding and independently select layers. Post-negotiation
|
||||
`getParameters()` can read the currently configured mode when one was
|
||||
requested, including a browser-selected replacement, but a successful request
|
||||
does not reveal whether the encoder is hardware or software.
|
||||
|
||||
The remaining capability signals cannot enforce the product's no-silent-
|
||||
software-fallback rule. Media Capabilities defines `powerEfficient` as an
|
||||
optimal-power judgment left to the user agent and explicitly notes that
|
||||
software can qualify. RTCStats exposes `encoderImplementation` and
|
||||
`powerEfficientEncoder` only when hardware exposure is allowed; the latter
|
||||
should reflect acceleration but may use other information. These fields are
|
||||
valuable diagnostics, not an affirmative portable hardware contract.
|
||||
|
||||
Official implementation surfaces checked 2026-08-19 are deliberately treated
|
||||
as boundaries, not UA quality rankings:
|
||||
|
||||
| Implementation | Static SVC surface |
|
||||
| --- | --- |
|
||||
| Chromium/Chrome | Chrome 111 shipped outgoing-track SVC selection; current Blink WebIDL exposes `scalabilityMode` behind its runtime feature. Exact codec/mode and hardware use remain runtime outcomes. |
|
||||
| Firefox | Current Firefox WebIDL omits `scalabilityMode`; Mozilla's implementation issue remains assigned and behind a preference. |
|
||||
| Safari/WebKit | Current WebKit WebIDL omits the standard member. LiveKit 2.22.0 carries a Safari-specific legacy encoding branch, but that is an SDK workaround rather than a portable standard setter/readback contract. |
|
||||
|
||||
Pinned LiveKit can select layers per SFU subscriber: client 2.22.0 exposes
|
||||
`RemoteTrackPublication.setVideoQuality()`, and server 1.13.5 maps each
|
||||
subscriber's quality/dimensions/FPS to maximum spatial/temporal layers on that
|
||||
subscriber's downtrack. The actual selector remains codec-dependent: VP8 has a
|
||||
temporal selector, whereas H.264/H.265 simulcast is spatial-only. That useful
|
||||
SFU contract still does not extend to direct/peer receivers. Screener's current
|
||||
SFU subscriber calls `setSubscribed(true)` and requests
|
||||
`setVideoQuality(HIGH)`. `HIGH` is a ceiling, not a guarantee that BWE forwards
|
||||
the high layer, so the application does not own the actual layer choice.
|
||||
|
||||
For this screen-share product the pin has an additional hard mismatch. Client
|
||||
2.22.0 overwrites SVC screen-share publication to `L1T3`, even when another
|
||||
mode was supplied: one spatial resolution and three temporal layers. It
|
||||
therefore supplies no low-resolution spatial base and does not satisfy the
|
||||
accepted LOW/HIGH contract. LiveKit also documents that Dynacast can pause only an entire SVC
|
||||
stream, not individual SVC layers. Changing those contracts would require a
|
||||
different dependency/native design decision, not a runtime proof of the pinned
|
||||
path. No SVC harness was written and no Chrome run was performed. None of the
|
||||
rejected standard paths may bypass PR #28's stock-GCC/RTX stop line.
|
||||
|
||||
## Why Offline Encoding Presets Do Not Transfer
|
||||
|
||||
[Maruko Toolbox](https://github.com/wzxjohn/marukotoolbox) is an Apache-2.0
|
||||
GUI around offline video-processing tools. Its x264-style CRF, slow presets,
|
||||
lookahead, and multi-pass workflows trade encoding time and buffering for file
|
||||
size and quality. Interactive WebRTC cannot buffer or revisit future frames in
|
||||
that way. The transferable principle is to treat bitrate, image detail, frame
|
||||
rate, and encode cost as a joint tradeoff; its concrete offline presets are not
|
||||
a WebRTC configuration baseline.
|
||||
The GUI repository is Apache-2.0, while x264 itself uses GPL-2.0 or commercial
|
||||
licensing; no implementation or preset is copied into Screener.
|
||||
|
||||
Discord's published Go Live design reaches the same broad conclusion for
|
||||
realtime game streaming: bandwidth estimates, frame delivery, image quality,
|
||||
latency, CPU, and memory conflict and must be balanced. Its native encoder
|
||||
tuning and hardware integration are not available to a browser-only sender and
|
||||
must not be presented as settings this project already has.
|
||||
|
||||
## Current Source Policy
|
||||
|
||||
The three user-visible profiles remain ceilings rather than promised rates:
|
||||
|
||||
| Profile | Capture ceiling | Sender bitrate ceiling |
|
||||
| --- | --- | ---: |
|
||||
| 1080p60 | 1920x1080 at 60 fps | 8 Mbps |
|
||||
| 1080p30 | 1920x1080 at 30 fps | 5 Mbps |
|
||||
| 720p30 | 1280x720 at 30 fps | 3 Mbps |
|
||||
|
||||
Current v12 source and production default to the middle `1080p30` ceiling.
|
||||
Choosing that
|
||||
default trades a 60 fps ceiling for a 1080p capture bound. The recommended set
|
||||
remains exactly the three profiles
|
||||
above. Both expose `480p` only as an advanced `854x480` resolution whose frame rate and bitrate are
|
||||
selected independently, not as a fourth profile or preset ID. LiveKit currently
|
||||
uses the same 1080p30 at 5 Mbps screen-share preset, but neither preset
|
||||
guarantees the emitted resolution, frame rate, or bitrate.
|
||||
|
||||
- Initial capture supplies `ideal` and `max` bounds for resolution and frame
|
||||
rate; the actual result is read back from `MediaStreamTrack.getSettings()`.
|
||||
- A live profile change uses `track.applyConstraints()` and updates every
|
||||
current sender with `RTCRtpSender.setParameters()`. It does not reopen the
|
||||
source picker or renegotiate healthy peer connections.
|
||||
- Current source and production leave the video hint unset so display capture
|
||||
retains browser screen semantics. Recommended profiles and the advanced
|
||||
initial value use `balanced`, with explicit `maintain-resolution` and
|
||||
`maintain-framerate` choices. None promises an emitted resolution or rate.
|
||||
- `maxBitrate` and `maxFramerate` are ceilings. They are neither minimums nor
|
||||
target guarantees, and the project does not use SDP bitrate hacks.
|
||||
- The strict `screener-v12` Share advanced settings panel in current source and
|
||||
production accepts 480p/720p/1080p/1440p, integer 15-60 fps, 2-12 Mbps, and
|
||||
the three preferences. Browser media is fixed VP8, with no codec UI,
|
||||
quality-state field, or wire field. The `480p` choice is only advanced
|
||||
`854x480`, not a fourth recommended profile. Its 64/128/256 kbps audio
|
||||
ceiling, default 128, is live-switchable on the existing Opus path. Display
|
||||
capture does not standardize channel-count or sample-rate control. The peer
|
||||
receive contract permits Opus `stereo=1;maxaveragebitrate=256000`, paired with pinned
|
||||
LiveKit's explicit high-quality stereo/forceStereo option; the selected sender
|
||||
ceiling remains separate from negotiated and observed bitrate.
|
||||
DTX stays fixed off, RED retains the pinned SDK default, and FEC remains
|
||||
browser/SDK negotiation, not a control or custom adaptation algorithm. The
|
||||
full boundary is recorded in `docs/research/browser-screen-audio-quality.md`.
|
||||
- Every sender update derives from `getParameters()`, calls `setParameters()`,
|
||||
then reads requested/applied bitrate, frame rate, scale, and preference.
|
||||
Rejection or browser rewriting is visible rather than console-only.
|
||||
- A P2P or browser-relay sender is constrained to VP8 before the first offer and
|
||||
receives its quality profile before that offer. Explicit source or
|
||||
profile changes use the existing serialized sender-mutation path; accepting
|
||||
an answer does not trigger an extra whole-profile write.
|
||||
- One strict room setting is last-wins for current/future peer relays and the
|
||||
configured SFU publisher. Ordinary P2P keeps that state local and does not add
|
||||
it to the authenticated wire.
|
||||
- Pausing sharing disables every track in the current capture stream, producing
|
||||
black video and silence without closing the room or media connection.
|
||||
|
||||
Current source and production stop at manual bounded controls.
|
||||
They add no composite score, periodic adjustment, runtime codec switching, SDP
|
||||
bitrate manipulation, or scene detector. Three consecutive samples of one
|
||||
non-`none` native `qualityLimitationReason` produce one explanatory warning; a
|
||||
reason change or recovery resets it and never triggers a media action.
|
||||
|
||||
A Chrome 151 loopback smoke with one host, three viewers, and synthetic 720p30
|
||||
video propagated balanced then clarity settings to every participant. Every
|
||||
baseline active outbound video sender displayed the matching requested/applied
|
||||
preference, peer-connection fingerprints stayed unchanged, and every viewer's
|
||||
decoded-frame and foreground `requestVideoFrameCallback` counters grew after
|
||||
each change. Host media edges peaked at two and relay edges at one. This verifies
|
||||
controls and continuity, not visual quality, full-resolution performance,
|
||||
CPU/GPU cost, public networks, or sustained behavior.
|
||||
|
||||
## Deliberate Non-Goals
|
||||
|
||||
- No canvas pixel-difference detector, machine-learned rate controller, or
|
||||
periodic profile switching.
|
||||
- No copied x264 CRF/preset recipe in the browser path.
|
||||
- Browser video is fixed VP8, with no runtime, automatic, or user-selected codec
|
||||
switching.
|
||||
- No channel-count, sample-rate, codec, arbitrary bitrate, stereo, DTX, RED or
|
||||
FEC control, and no inference of actual stereo or sample rate from
|
||||
`opus/48000/2`. Screen media uses one route-consistent stereo contract;
|
||||
microphone voice, if ever accepted, remains a separate track and policy.
|
||||
- No promise that all browsers honor `applyConstraints` or degradation
|
||||
preference identically.
|
||||
- No automatic composite quality score or periodic profile controller before
|
||||
the controlled production capture identifies a reproducible bottleneck.
|
||||
- No room-wide minimum-of-viewers target, per-viewer encoder, UA/device quality
|
||||
ranking, or silent software SVC fallback.
|
||||
|
||||
## Verification Gate
|
||||
|
||||
Use the same static UI scene and deterministic high-motion game scene at
|
||||
720p30, 1080p30, and 1080p60. Correlate A capture, B outbound, and C inbound at
|
||||
the same interval. Record actual dimensions/FPS/bitrate, limitation reason,
|
||||
codec, encoder implementation, interval encode/decode cost, drops/freezes,
|
||||
jitter, RTT, loss/retransmission, selected direct/SFU path, and host CPU/GPU.
|
||||
A live profile change must preserve peer connection IDs, avoid a second source
|
||||
prompt, and visibly converge to the requested bounds.
|
||||
|
||||
Compute per-frame encode and decode cost from adjacent samples of cumulative
|
||||
`totalEncodeTime`/`totalDecodeTime` and frame counters. The first sample, a
|
||||
zero-frame interval, changed stats object, or a counter reset is unknown and
|
||||
establishes a new baseline. Skip overlapping sampling ticks on the same peer so
|
||||
reports cannot complete out of order. Do not use a connection-lifetime average
|
||||
to judge a later overload. Use browser WebRTC diagnostics plus a small run
|
||||
manifest for time series and percentiles rather than adding a server telemetry
|
||||
pipeline.
|
||||
|
||||
Compare image readability and motion continuity instead of declaring success
|
||||
from FPS alone. If reproducible evidence later shows a supported preference
|
||||
oscillates or makes the wrong tradeoff on supported machines, revise that
|
||||
explicit option only from a new controlled gate. The exact Browser SFU gate
|
||||
showed one VP8 outbound video encoding, no `q,h` simulcast, and continuing
|
||||
publisher/subscriber frame progress without changing ADR-0005 resource accounting.
|
||||
The application's evidence windows remain diagnostic only; they do not command
|
||||
another representation, change topology, or override stock congestion control.
|
||||
Current parent selection hard-filters authorization, source reachability,
|
||||
acyclicity, capacity, and resource admission, then orders eligible P2P parents
|
||||
by resulting depth, remaining sender capacity, stable join order, and peer
|
||||
identity before SFU. It has no network-quality score.
|
||||
|
||||
A manual media reconnect stays on the current exact route: P2P rebuilds the
|
||||
same parent connection, and SFU reconstructs the same subscription. Browser
|
||||
page refresh also rebinds the same stable participant and committed graph.
|
||||
Only actual recovery exhaustion enters the normal route-failure operation and
|
||||
tries other eligible P2P parents before SFU.
|
||||
|
||||
Quality-driven relay abdication or active parent switching would require
|
||||
counterfactual alternative-path evidence, bounded probing, and a separate
|
||||
ADR-0005 decision. It is not implemented or authorized by current diagnostics.
|
||||
|
||||
## Current Verification Gaps
|
||||
|
||||
- Real games under competing CPU/GPU load at 720p30, 1080p30, and 1080p60.
|
||||
- Weaker Windows/macOS/Linux Hosts and actual VP8 encoder implementation data.
|
||||
- Mixed P2P/SFU sessions on heterogeneous public networks, including recovery.
|
||||
- Mobile foreground/background playback, page reclamation, and relay survival.
|
||||
- Long-running encode/decode cost, thermals, A/V synchronization, and resource
|
||||
admission at the 20-Viewer bound.
|
||||
|
||||
Measurements must correlate capture, outbound, and inbound stats over the same
|
||||
interval. Missing counters and identity changes remain unknown, not zero.
|
||||
|
||||
## Primary Sources
|
||||
|
||||
- [W3C MediaStreamTrack Content Hints](https://www.w3.org/TR/mst-content-hint/)
|
||||
- [W3C Screen Capture](https://www.w3.org/TR/screen-capture/)
|
||||
- [W3C Media Capture and Streams `getSettings()`](https://www.w3.org/TR/mediacapture-streams/#dom-mediastreamtrack-getsettings)
|
||||
- [W3C WebRTC](https://www.w3.org/TR/webrtc/)
|
||||
- [W3C WebRTC codec preferences](https://www.w3.org/TR/webrtc/#dom-rtcrtptransceiver-setcodecpreferences)
|
||||
- [W3C WebRTC sender track replacement](https://www.w3.org/TR/webrtc/#dom-rtcrtpsender-replacetrack)
|
||||
- [Jitsi codec preference and renegotiation](https://jitsi.github.io/lib-jitsi-meet/classes/JitsiConference._internal_.JingleSessionPC.html)
|
||||
- [RFC 7742 WebRTC video codec requirements](https://www.rfc-editor.org/rfc/rfc7742.html)
|
||||
- [Via LA AVC/H.264 licensing program](https://via-la.com/licensing-programs/avc-h-264/)
|
||||
- [W3C WebRTC Statistics](https://www.w3.org/TR/webrtc-stats/)
|
||||
- [W3C WebRTC SVC](https://www.w3.org/TR/webrtc-svc/)
|
||||
- [W3C Media Capabilities](https://www.w3.org/TR/media-capabilities/)
|
||||
- [W3C WebCodecs](https://www.w3.org/TR/webcodecs/)
|
||||
- [Chrome 111 WebRTC SVC extension](https://developer.chrome.com/blog/chrome-111-beta)
|
||||
- [Chromium `RTCRtpEncodingParameters` WebIDL](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/third_party/blink/renderer/modules/peerconnection/rtc_rtp_encoding_parameters.idl)
|
||||
- [Firefox `RTCRtpEncodingParameters` WebIDL](https://searchfox.org/firefox-main/source/dom/webidl/RTCRtpParameters.webidl)
|
||||
- [Firefox WebRTC-SVC implementation status](https://bugzilla.mozilla.org/show_bug.cgi?id=1571470)
|
||||
- [WebKit `RTCRtpEncodingParameters` WebIDL](https://github.com/WebKit/WebKit/blob/main/Source/WebCore/Modules/mediastream/RTCRtpEncodingParameters.idl)
|
||||
- [RFC 6184 H.264 RTP payload format](https://www.rfc-editor.org/rfc/rfc6184.html)
|
||||
- [RFC 7741 VP8 RTP payload format](https://www.rfc-editor.org/rfc/rfc7741.html)
|
||||
- [RFC 9628 VP9 RTP payload format](https://www.rfc-editor.org/rfc/rfc9628.html)
|
||||
- [IANA AV1 media type and format parameters](https://www.iana.org/assignments/media-types/video/AV1)
|
||||
- [Chromium/libwebrtc video stats origins](https://webrtc.googlesource.com/src/+/HEAD/video/g3doc/stats.md)
|
||||
- [Chromium WebRTC hardware encoder factory](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/third_party/blink/renderer/platform/peerconnection/rtc_video_encoder_factory.cc)
|
||||
- [Chromium Windows H.264 constrained-baseline acceleration gate](https://github.com/chromium/chromium/blob/main/third_party/blink/renderer/platform/peerconnection/webrtc_util.cc)
|
||||
- [Chromium Windows Media Foundation encoder selection](https://chromium.googlesource.com/chromium/src/+/master/media/gpu/windows/media_foundation_video_encode_accelerator_win.cc)
|
||||
- [Chromium WebRTC hardware fallback and simulcast initialization](https://chromium.googlesource.com/chromium/src/+/HEAD/third_party/blink/renderer/platform/peerconnection/rtc_video_encoder.cc)
|
||||
- [Chromium WebRTC AV1 hardware feature gate](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/media/webrtc/webrtc_features.cc)
|
||||
- [MDN `RTCRtpSender.setParameters()`](https://developer.mozilla.org/en-US/docs/Web/API/RTCRtpSender/setParameters)
|
||||
- [Chromium `motion` to libwebrtc `kFluid` bridge](https://chromium.googlesource.com/chromium/src/third_party/+/refs/heads/main/blink/renderer/modules/peerconnection/media_stream_video_webrtc_sink.cc)
|
||||
- [libwebrtc `motion`/`kFluid` sender classification](https://webrtc.googlesource.com/src/+/3b1eab8a69cb5078befb021c5492d3f204a7d6a2/pc/rtp_sender.cc)
|
||||
- [libwebrtc pre-negotiation sender parameter storage](https://webrtc.googlesource.com/src/+/refs/heads/main/pc/rtp_sender.cc)
|
||||
- [libwebrtc encoder content-type selection](https://webrtc.googlesource.com/src/+/9caef2a8b88f389af10cee841732c42a98d3d45d/media/engine/webrtc_video_engine.cc)
|
||||
- [libwebrtc conditional screen-share degradation mapping](https://webrtc.googlesource.com/src/+/refs/heads/main/video/video_stream_encoder.cc)
|
||||
- [MediaStreamTrack Content Hints](https://www.w3.org/TR/mst-content-hint/)
|
||||
- [Screen Capture](https://www.w3.org/TR/screen-capture/)
|
||||
- [WebRTC](https://www.w3.org/TR/webrtc/)
|
||||
- [WebRTC Statistics](https://www.w3.org/TR/webrtc-stats/)
|
||||
- [libwebrtc adaptation overview](https://webrtc.googlesource.com/src/+/HEAD/video/g3doc/adaptation.md)
|
||||
- [libwebrtc per-send-stream encoder construction](https://webrtc.googlesource.com/src/+/refs/heads/main/video/video_send_stream_impl.cc)
|
||||
- [libwebrtc removal of the automatic animation-detection experiment, 2024-05-22](https://webrtc.googlesource.com/src/+/1d7d0e6e2c5002815853be251ce43fe88779ac85)
|
||||
- [LiveKit screen-share presets](https://github.com/livekit/client-sdk-js/blob/main/src/room/track/options.ts)
|
||||
- [LiveKit degradation defaults](https://github.com/livekit/client-sdk-js/blob/main/src/room/participant/publishUtils.ts)
|
||||
- [LiveKit screen-share encoding construction](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/participant/publishUtils.ts)
|
||||
- [LiveKit video simulcast and Dynacast](https://docs.livekit.io/transport/media/advanced/)
|
||||
- [LiveKit selective subscription](https://docs.livekit.io/transport/media/subscribe/)
|
||||
- [LiveKit client 2.22.0 room defaults](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/defaults.ts)
|
||||
- [LiveKit client 2.22.0 room and Dynacast options](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/options.ts)
|
||||
- [LiveKit client 2.22.0 room option merge](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/Room.ts)
|
||||
- [LiveKit client 2.22.0 SVC defaults](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/track/options.ts)
|
||||
- [LiveKit client 2.22.0 screen-share SVC, Dynacast enablement, and republish lifecycle](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/participant/LocalParticipant.ts)
|
||||
- [LiveKit client 2.22.0 start-bitrate negotiation](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/PCTransport.ts)
|
||||
- [LiveKit client 2.22.0 SVC encoding construction](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/participant/publishUtils.ts)
|
||||
- [LiveKit client 2.22.0 subscriber quality control](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/track/RemoteTrackPublication.ts)
|
||||
- [LiveKit server 1.13.5 per-subscriber layer application](https://github.com/livekit/livekit/blob/v1.13.5/pkg/rtc/subscribedtrack.go)
|
||||
- [LiveKit server 1.13.5 codec-specific layer selectors](https://github.com/livekit/livekit/blob/v1.13.5/pkg/sfu/forwarder.go)
|
||||
- [LiveKit client 2.22.0 Dynacast and saved degradation preference](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/track/LocalVideoTrack.ts)
|
||||
- [LiveKit client 2.22.0 local sender stats](https://github.com/livekit/client-sdk-js/blob/v2.22.0/src/room/track/LocalTrack.ts)
|
||||
- [LiveKit server 1.13.5 Dynacast quality aggregation](https://github.com/livekit/livekit/blob/v1.13.5/pkg/rtc/dynacast/dynacastqualityvideo.go)
|
||||
- [LiveKit server 1.13.5 enabled-quality generation](https://github.com/livekit/livekit/blob/v1.13.5/pkg/rtc/dynacast/dynacastmanagervideo.go)
|
||||
- [LiveKit server 1.13.5 release assets and checksums](https://github.com/livekit/livekit/releases/tag/v1.13.5)
|
||||
- [Microsoft `New-NetQosPolicy`](https://learn.microsoft.com/en-us/powershell/module/netqos/new-netqospolicy)
|
||||
- [Linux network namespaces](https://man7.org/linux/man-pages/man7/network_namespaces.7.html)
|
||||
- [Linux `tc-tbf`](https://man7.org/linux/man-pages/man8/tc-tbf.8.html), [`tc-netem`](https://man7.org/linux/man-pages/man8/tc-netem.8.html), and [`tc-flower`](https://man7.org/linux/man-pages/man8/tc-flower.8.html)
|
||||
- [Jitsi desktop degradation preference](https://github.com/jitsi/lib-jitsi-meet/blob/master/modules/RTC/TraceablePeerConnection.ts)
|
||||
- [Discord Go Live architecture](https://discord.com/blog/how-it-all-goes-live-an-overview-of-discords-streaming-technology)
|
||||
- [Discord encoder-quality case study](https://discord.com/blog/from-blocky-to-brilliant-improving-video-quality-on-discord-go-live-on-amd-gpus)
|
||||
- [NVIDIA NVENC application note](https://docs.nvidia.com/video-technologies/video-codec-sdk/13.1/nvenc-application-note/index.html)
|
||||
- [Agora Windows screen sharing](https://doc.shengwang.cn/doc/rtc/windows/basic-features/screen-share)
|
||||
- [Agora Windows encoding preference](https://doc.shengwang.cn/api-ref/rtc/windows/API/enum_encodingpreference)
|
||||
- [Oopz help center](https://help.oopz.cn/)
|
||||
- [LiveKit server forwarding](https://github.com/livekit/livekit/blob/v1.13.5/pkg/sfu/forwarder.go)
|
||||
- [Chromium WebRTC encoder factory](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/third_party/blink/renderer/platform/peerconnection/rtc_video_encoder_factory.cc)
|
||||
@@ -160,7 +160,7 @@ parameters.degradationPreference = "balanced";
|
||||
await sender.setParameters(parameters);
|
||||
```
|
||||
|
||||
这些参数是偏好或上限，不能绕过浏览器拥塞控制，也不能保证目标码率。`balanced` 允许浏览器在分辨率和帧率之间权衡，但不规定具体算法；视频 track 不设置 `contentHint`，让 Chromium/libwebrtc 保留 display capture 的 screen 分类。当前源码链与测量要求见 [Realtime quality adaptation](./realtime-quality-adaptation.md)。静态画面通常能降低编码数据量，但规范不保证所有浏览器主动降低捕获频率或 GPU 开销。
|
||||
这些参数是偏好或上限，不能绕过浏览器拥塞控制，也不能保证目标码率。`balanced` 允许浏览器在分辨率和帧率之间权衡，但不规定具体算法；视频 track 使用标准 `contentHint = "motion"` 表达游戏动态意图。当前源码链与测量要求见 [Realtime quality adaptation](./realtime-quality-adaptation.md)。静态画面通常能降低编码数据量，但规范不保证所有浏览器主动降低捕获频率或 GPU 开销。
|
||||
|
||||
因此首版不实现画面差分检测、周期性 `applyConstraints()` 或自定义动态 FPS 状态机。先对静态桌面和高动态游戏分别记录 `framesEncoded`、发送码率、`totalEncodeTime / framesEncoded`、`qualityLimitationReason` 及主机 CPU/GPU 占用；只有测量显示浏览器行为留下显著问题时，再设计最小的控制策略。这避免用额外竞态、计时器和画质跳变解决一个可能已由捕获器、编码器和 WebRTC 拥塞控制处理的问题。
|
||||
|
||||
|
||||
+17
-71
@@ -2,82 +2,28 @@
|
||||
|
||||
Last updated: 2026-08-25
|
||||
|
||||
This file is a bootstrap prompt, not a product-truth owner. The linked documents
|
||||
replace any duplicated or conflicting statement here. Update or remove this file
|
||||
after the handoff boundary changes.
|
||||
This file is only a bootstrap pointer. Product truth lives in the linked owners.
|
||||
|
||||
## Prompt
|
||||
## Resume Prompt
|
||||
|
||||
```text
|
||||
Continue the Screener project from the canonical repository root:
|
||||
Continue Screener from the canonical repository root:
|
||||
C:\Users\TNTcraft\Documents\GitHub\Screener
|
||||
|
||||
Before changing code, deployment, branches, worktrees, or TODO state:
|
||||
Read AGENTS.md and the complete repo-tracked stop-that-shit skill first. Then
|
||||
read CONTRIBUTING.md, docs/project-memory.md, docs/status.md, docs/todo.md, the
|
||||
relevant requirement/design/ADR/research owner, and inspect branch/HEAD, status,
|
||||
staged diff, and worktrees. Canonical main and repo truth override old branches,
|
||||
worktrees, handoffs, agent memory, and chat summaries.
|
||||
|
||||
1. Read the root AGENTS.md instructions supplied for this repository, then read
|
||||
CONTRIBUTING.md, docs/project-memory.md, docs/status.md, docs/todo.md,
|
||||
docs/需求理解.md, docs/方案设计.md, docs/maintenance.md, and the relevant
|
||||
ADR/research owner.
|
||||
2. Inspect git status, current branch/HEAD, worktree list, and the diff. Preserve
|
||||
every pre-existing edit. The canonical root/main owns truth; auxiliary
|
||||
worktrees, branches, agent memories, chat summaries, and old TODOs do not.
|
||||
3. Distinguish accepted product truth, current source, and current production.
|
||||
Do not report an accepted design as implemented or deployed.
|
||||
4. Resolve meaning and update all affected truth owners before implementation.
|
||||
Keep the model simple, use existing framework behavior, avoid case-by-case
|
||||
patches, and do not add compatibility for unpublished internal releases.
|
||||
Current production and main identity is owned by docs/status.md. ADR-0002 owns
|
||||
memory rooms; ADR-0005 owns automatic direct/peer-to-LiveKit routing; ADR-0007
|
||||
owns framework media adaptation. Current routes are sticky and quality evidence
|
||||
is diagnostic only. Browser video is VP8 with contentHint=motion; LiveKit owns
|
||||
SFU representations; Screener configures no TURN or media TCP.
|
||||
|
||||
Current high-signal boundary:
|
||||
|
||||
- docs/status.md owns the exact source/production/hold state.
|
||||
- docs/todo.md is the only executable work ledger and owns ordering.
|
||||
- ADR-0005 owns the accepted automatic route model: direct/STUN peer followed
|
||||
only by the dedicated LiveKit SFU/UDP fallback. Browser SFU publisher and
|
||||
subscriber PCs explicitly use no external ICE server and check the dedicated
|
||||
public SFU candidate directly; ordinary peers and LiveKit server-side public
|
||||
IP discovery retain deployment STUN. Production exact deployed
|
||||
application/runtime revision `1d8761528d0dba43fb6d818df3934483ba2f5340`,
|
||||
release `1d87615`, uses strict `screener-v12` and includes exact Browser SFU
|
||||
ICE-server-isolation implementation
|
||||
`ae09c760adec76fd26da611d4928486d105c6d3b`. Chrome 151 on that exact production
|
||||
verified one no-RID publisher encoding and progressing publisher/subscriber
|
||||
frames over LiveKit UDP. The route uses one short-lived
|
||||
exact-generation decoded-frame observer only for the pending candidate inside
|
||||
the existing total deadline; the periodic stats sampler still owns active-path
|
||||
diagnostics and stalls. Broader external-network and SFU lifecycle evidence
|
||||
remains open. Do not run executable/native suites unless their later acceptance
|
||||
boundary specifically requires them.
|
||||
- ADR-0002 owns the room model. Current source and production use strict v12 with
|
||||
exactly `open | private` code entry and the room-lived 22-character Viewer
|
||||
grant. Neither source nor production uses SQLite; exact release operations and
|
||||
change-scoped infrastructure recovery are owned by deployment.
|
||||
- Browser display video must leave `contentHint` unset; audio keeps `music`.
|
||||
Browser video is fixed VP8 across direct, browser-relay, and SFU paths; strict
|
||||
v12 source and production have no codec UI/state/wire or fallback media codec.
|
||||
Automated source gates and the exact deployment postflight passed. Chrome 151
|
||||
physically verified direct/browser-relay/SFU screen-audio continuity and all
|
||||
three sender ceilings. An exact-production VP8 gate sustained about 59.5 fps
|
||||
on direct and Browser-relay paths and rejected the then-deployed `bf32859`
|
||||
SFU `q,h` publication because active `LOW` reduced `HIGH`. ADR-0007 accepts
|
||||
LiveKit-owned half-resolution plus original-resolution simulcast, Dynacast,
|
||||
and server send-side BWE. Current source implements that combination while
|
||||
production remains single-encoding; Chrome
|
||||
151 verified the production no-RID baseline and separately proved the accepted
|
||||
native adaptation on the pinned stack.
|
||||
- Ordinary browser ICE owns direct reachability. Do not build port prediction,
|
||||
guessed candidates, NAT classification, TCP probes, quality scores, or
|
||||
quality-driven reparenting. Current-path quality is diagnostic; a healthy
|
||||
decoded edge remains sticky.
|
||||
- docs/maintenance.md owns the reusable mechanism-simplification review;
|
||||
ADR-0002 owns the accepted memory-room tradeoff. The later repository audit in
|
||||
TODO is not permission for broad cleanup during this handoff.
|
||||
- The canonical root/main checkpoint reconciles the route truth audit with this
|
||||
room decision. Use that coherent truth as the base; do not replace it with
|
||||
older documents from another worktree.
|
||||
- Existing auxiliary worktrees and branches are retained candidates or audit
|
||||
material. Do not merge or delete them until their unique changes, references,
|
||||
and reparse/junction safety are checked against current main.
|
||||
|
||||
Follow the current ledger in order and report each verified physical-evidence
|
||||
boundary without reviving stale agent or branch facts.
|
||||
Follow docs/todo.md in order. Do not run Native/executable work, broad repository
|
||||
cleanup, quality-driven reparenting, or another transport unless its decision
|
||||
boundary is explicitly opened. Preserve unique or dirty worktrees until merged-
|
||||
head, reference, and reparse audits prove cleanup safe.
|
||||
```
|
||||
+60
-29
@@ -2,47 +2,78 @@
|
||||
|
||||
Last updated: 2026-08-25
|
||||
|
||||
This is the current execution index. Git history owns completed timelines; [verification status](./verification-status.md) owns evidence boundaries.
|
||||
This is the current execution index. Git history owns completed timelines;
|
||||
[verification status](./verification-status.md) owns evidence boundaries.
|
||||
|
||||
## Production
|
||||
|
||||
- `https://share.bonfire.icu` runs exact deployed application/runtime revision `be53c4d5794d38d5d406c876309958d7117ee601`, release `be53c4d`, wire `screener-v12`, from `/opt/screener/releases/be53c4d`; that revision is integrated in canonical `main`. Its immutable runtime tar SHA-256 is `5c174c30ffacd98738a364b825b915b84432823c261272684937361be2451144`, and its 39-file manifest SHA-256 is `c34dba43a25c6b8447e1153a5be3191bf82cff94117ed31edc325d3f7cf8541f`.
|
||||
- `screener`, LiveKit, coturn, and nginx are active/running; local/public `/healthz` and the served Browser asset match the immutable release, and all four services report `NRestarts=0`. The main Browser asset is `assets/index-BKEVI7Ux.js` with SHA-256 `22114fc8a26b3f03f37973555433d1e73d52d1c7b76fa3c7cc597500823bd62d`; predecessor asset `assets/index-Bjcb0_oa.js` returns 404.
|
||||
- The service process resolves its working directory to exact release `be53c4d`; it has no writable room `StateDirectory` or SQLite runtime.
|
||||
- Production admits one Host plus 20 Viewers, applies endpoint capacity `2`, rejects stale Browser and executable-sender wires before room authority, and runs the one-controller exact-candidate route model with one staged total deadline. Rooms use random free four-digit codes, a 24-hour dormant lease, restart loss, local Host creation preferences, a room-lived 22-character Viewer grant, and exactly `open | private` code entry. Private rooms without a password are invitation-only; configuring one additionally permits matching code-and-password entry. LiveKit remains dedicated with `room.auto_create: false`, `max_participants: 21`, and global admission `1` publication ingress / `20` subscription egress.
|
||||
- Production media ports remain STUN-only UDP 3478 and LiveKit UDP 7882; coturn TCP/TLS, port 5349, media TCP, and relay ranges are disabled. The public Web ingress remains TCP 80/443; Node 8787 and LiveKit control/signaling 7880 are internal-only. The stable `inet bonfire_filter` table SHA-256 is `afaf9d066e9f292d8bd19032b38c0978ccf60dd01c2fe7f210968ff751f90534`; the dynamic fail2ban table is excluded from that identity.
|
||||
- SFU subscribers reconcile Host screen publications across connect, activation, participant arrival, track publication, and reconnect. LiveKit runs at `warn`/Pion `error`, and the Screener unit waits boundedly for the same-host LiveKit control listener before startup. The decoded-progress cadence correction is deployed; the earlier subscriber-PC closure remains only the reproduction, so a controlled recovery transaction and physical network-loss evidence remain open.
|
||||
- Production Browser SFU publisher and subscriber PCs explicitly use empty external ICE-server lists while retaining LiveKit-signaled UDP candidates. The owner verified that the deployed isolation fixes the reported SFU connection failure; the prior P1 core-route hold is closed.
|
||||
- Production leaves video `contentHint` unset, keeps audio `contentHint = "music"`, and fixes Browser direct, browser-relay, and SFU video to VP8 without codec UI, quality state, wire fields, or fallback media codecs. Its SFU publisher supplies LiveKit one half-actual-resolution lower preset capped at 30 fps with proportional bitrate beside the original representation, enables Dynacast and server send-side BWE, keeps AdaptiveStream disabled, and adds no Screener layer selector.
|
||||
- `https://share.bonfire.icu` runs exact application/runtime revision
|
||||
`af348ee1d508a3af02b18a7f46c461953798e19d`, release `af348ee`, wire
|
||||
`screener-v12`, from `/opt/screener/releases/af348ee`. The immutable runtime
|
||||
tar SHA-256 is
|
||||
`4922cc31d7caaad7c5412b1f2fe1d73f788d16d92386efad6eac358a97b8a78e`;
|
||||
its 39-file manifest SHA-256 is
|
||||
`5b397809b664b9a55885ac603df615b537bf51276e7274416820870956242d7d`.
|
||||
- The served Browser entry references `assets/index-BvuZY6Hc.js`; the public
|
||||
asset is 488473 bytes with SHA-256
|
||||
`e8fc0500496ee101031a9cfebfcc22d3b69e66d2cac5e875448b56620dbb21c5`.
|
||||
Public `/healthz` returns 200. The release postflight found Screener, LiveKit,
|
||||
coturn, and nginx active with zero restarts.
|
||||
- Production uses process-memory four-digit rooms, a 24-hour dormant lease,
|
||||
room-lived 22-character grants, `open | private` code entry, one Host plus 20
|
||||
Viewers, endpoint copy cap `2`, and no SQLite runtime.
|
||||
- Public media listeners remain STUN-only UDP 3478 and LiveKit UDP 7882. Web
|
||||
ingress is TCP 80/443; Node 8787 and LiveKit control/signaling 7880 are
|
||||
private. TURN, ICE/TCP, media TCP, TLS relay, port 5349, and relay ranges are
|
||||
disabled.
|
||||
- Browser video is fixed VP8 with `contentHint = "motion"`; audio uses `music`.
|
||||
The SFU publisher leaves representation construction to pinned LiveKit,
|
||||
enables Dynacast, uses server send-side BWE, keeps AdaptiveStream disabled,
|
||||
and configures no external ICE servers on Browser SFU PCs.
|
||||
- Screen audio provides live 64/128/256 kbps ceilings with 128 default. SFU
|
||||
publication uses stereo, DTX off, and RED off. Early autoplay presentation is
|
||||
gated by current media connection state.
|
||||
|
||||
## Current Source
|
||||
|
||||
- Canonical root `main` is the integration truth; auxiliary branches and older worktrees do not supersede it. Current source and production use the single strict Browser `screener-v12` wire and reject every other wire before room authority.
|
||||
- Current source implements one Host plus up to `20` Viewers, one steady outbound media-copy cap for every non-server endpoint (default `2`, static `1/2/3`), and the process-memory room model with random four-digit codes, a default 24-hour dormant lease, exact-Host resume, restart loss, local Host preference replay, exactly `open | private` code entry, and no SQLite runtime. Every room defaults to `open` without a room password and independently receives a Viewer grant: one 22-character opaque value bound to that room incarnation. Private rooms without a password are invitation-only; configuring a password additionally permits matching code-only entry. Unallocated/reclaimed/expired code-only entry returns `ROOM_NOT_FOUND` without a password prompt.
|
||||
- One event-driven controller owns the committed graph and at most one room-serial child operation. It uses one deterministic candidate list/cursor and one total direct-then-SFU deadline; exact admission and physical resources remain charged through drain, and the pending candidate commits only after an exact-generation decoded-frame proof. A short-lived 100 ms stats observer proves only that candidate inside the existing deadline. The active-path 2-second sampler now keeps its existing 15-second decoded-progress cadence when an SFU subscriber has no video track, no merged report, or a failed stats read, without adding a timer or fabricating metrics. Healthy edges remain sticky and current-path quality does not authorize reparenting.
|
||||
- Ordinary peer ICE is STUN-only and the only application fallback is the dedicated LiveKit SFU over UDP. SFU generations use explicit no-default ingress/egress capacities, exact `reserved | committed | draining` accounting, delete-plus-absence release, one Host publication, and exact Viewer subscription handles. No Screener TURN, ICE/TCP, media TCP, or TLS-relayed media route exists.
|
||||
- Current Browser SFU publisher and subscriber connects supply fresh `autoSubscribe: false` plus `rtcConfig: { iceServers: [] }` options. Ordinary peers retain deployment STUN; LiveKit signaling candidates, server-side public-IP discovery, route deadlines, ports, and retry behavior are unchanged.
|
||||
- Current source leaves video `contentHint` unset and keeps audio `contentHint = "music"`. Browser direct and browser-relay offers contain VP8 as their only media codec. SFU publication fixes VP8 with no backup codec and supplies LiveKit one half-actual-resolution lower preset capped at 30 fps with proportional bitrate beside the original representation; Dynacast, the default maximum subscriber ceiling, server send-side BWE, and disabled AdaptiveStream remain unchanged. Codec UI, quality state, wire fields, fallback media codecs, and share-lifetime codec branches are absent. Live 64/128/256 kbps audio-ceiling mutation remains unchanged; ordinary Host and SFU publisher metrics select the exact current sender, relays reconcile audio-track additions and removals on the persistent stream, and an SFU reconnect reacquires the exact current publication/sender, avoids unchanged parameter/stat resets, or fails the route closed.
|
||||
- Current source implements typed first-frame Viewer presentation and recovery, Chinese user-facing route/access failures, local connection details without a diagnostic-file export, authenticated Host-only on-demand route snapshots for automatic acceptance, explicit missing-room presentation, and responsive entry controls. Host invitation controls remain independent from two-state room-code entry.
|
||||
- Recommended quality remains exactly `1080p60`, `1080p30`, and `720p30`; current source defaults to `1080p30`. Advanced resolution adds `480p` as `854x480` without adding a fourth preset, and advanced FPS and bitrate remain independent.
|
||||
- The deployed application tree passed repository hygiene, TypeScript, all 613 Web tests in 44 files, both production builds, two independent reviews, and immutable artifact/postflight checks. Earlier exact-production gates exercised direct, browser-relay, SFU audio continuity and all three audio ceilings, and sustained about 59.5 fps on direct and browser-relay paths; the pinned-stack gate proved native two-representation selection. The new lower representation's mixed-route quality effect still needs production media evidence. Exact applicability remains in [verification status](./verification-status.md).
|
||||
- Canonical root `main` is clean at the same exact `af348ee` revision as
|
||||
production. Auxiliary branches and worktrees do not supersede it.
|
||||
- One event-driven controller owns the committed graph and one room-serial child
|
||||
operation. It uses exact candidate identity, first-decoded-frame commit,
|
||||
strict rollback revisions, direct-then-SFU total deadline, and typed endpoint
|
||||
plus SFU resource accounting. Healthy decoded edges remain sticky.
|
||||
- WebRTC/LiveKit own media adaptation. Screener has no custom SFU layer list,
|
||||
quality score, layer selector, periodic rebalancing, or quality-driven parent
|
||||
change. P2P and SFU recover their current route before actual failure enters
|
||||
normal reassignment.
|
||||
- The staged `fix/viewer-page-lifecycle` candidate is based on exact `af348ee`.
|
||||
It rebaselines Viewer decoded-stall timing after page resume, makes a new
|
||||
current-generation frame clear stale media recovery state, uses native Viewer
|
||||
video controls, removes Host preview controls, and makes manual P2P reconnect
|
||||
rebuild the same parent. It is accepted but not yet committed, merged, or
|
||||
deployed.
|
||||
|
||||
## Current Milestone
|
||||
|
||||
1. Verify ADR-0007's deployed lightweight half-resolution representation in a mixed P2P/SFU room, then repeat the controlled active-SFU recovery transaction on exact production and finish representative ICE/STUN/SFU acceptance including screen-audio continuity and real-game A/V sync on mobile and heterogeneous networks. Browser port prediction, NAT classification, TCP probing, fake page keepalive, and quality-driven reparenting remain outside the accepted model.
|
||||
1. Integrate, validate, deploy, and postflight `fix/viewer-page-lifecycle`.
|
||||
2. Finish representative public-network direct, peer-relay, SFU, recovery,
|
||||
Pause/Resume, screen-audio, real-game A/V, mobile lifecycle, and all-UDP-
|
||||
blocked acceptance without adding another transport or quality controller.
|
||||
|
||||
## Active Boundaries
|
||||
|
||||
- `fix/configurable-relay-cap` is an old, incomplete draft and must not be merged as-is.
|
||||
- The desktop Host background/minimized report is deferred until a current-production real-game reproduction supplies synchronized media and CPU/GPU evidence. Mobile Viewer background playback and relay survival remain a separate Accepted Later lifecycle gate.
|
||||
- Open PR #192 and the Native stack are evidence/research, not pending product releases. Native senders, capture helpers, shared-encode executables, and their test binaries are outside the current Browser milestone and are not built or run.
|
||||
- Deployed surfaces and retained candidates that still need product decisions are indexed only in [the TODO ledger](./todo.md); do not extend or roll them back automatically.
|
||||
- Room lifetime, authorization/storage semantics, restart loss, and routing are aligned on strict v12 in current source and production. The application-only cutover reused unchanged infrastructure and configuration and does not maintain a full rollback/configuration backup; any future infrastructure or irreversible-state change requires recovery scoped to the surfaces it actually changes.
|
||||
- Real SFU recovery, heterogeneous networks, mobile lifecycle, audio/A-V device behavior, and endurance/resource measurements remain external acceptance evidence, not blockers for unrelated reversible work.
|
||||
- Desktop Host background capture remains unconfirmed. The current-browser
|
||||
screening did not reproduce a Host-page drop; accepted Viewer lifecycle work
|
||||
prevents frozen JavaScript wall time from becoming an immediate route failure
|
||||
but is not capture keepalive.
|
||||
- Current-path quality is diagnostic. High loss, RTT, jitter, low bitrate,
|
||||
resolution, FPS, or freeze counters do not trigger relay abdication or parent
|
||||
switching.
|
||||
- Native/executable senders, shared encode, distribution packages, broad UI
|
||||
polish, bilingual support, and repository-wide simplification are outside the
|
||||
current release.
|
||||
|
||||
## Current Hold
|
||||
|
||||
The active-SFU cadence correction is deployed, but the controlled recovery
|
||||
transaction has not yet been repeated on this release. That exact proof remains
|
||||
the P1 hold for route acceptance. Native/executable work and broad repository
|
||||
cleanup remain outside the current evidence boundary.
|
||||
No source or deployment P0 is open. Representative real-network and mobile
|
||||
physical evidence remains required before route acceptance is complete.
|
||||
+70
-28
@@ -2,45 +2,87 @@
|
||||
|
||||
Last reviewed: 2026-08-25
|
||||
|
||||
Only items in **Now** are executable after their stated decision gate. A branch name, unchecked requirement, experiment, review suggestion, or deployed behavior is not a TODO by itself.
|
||||
Only **Now** is executable. A branch, old experiment, observation, or accepted
|
||||
later topic is not implementation authority by itself.
|
||||
|
||||
## Now
|
||||
|
||||
1. **Verify deployed constrained SFU publication quality.** Production keeps the two LiveKit-owned VP8 spatial representations, Dynacast, send-side BWE, and no AdaptiveStream. It derives the half-actual-resolution lower publication preset at no more than 30 fps with proportional bitrate and retains that option for LiveKit-owned source replacement or republish; it does not mutate LOW behind the SDK's Dynacast lock. In the same mixed P2P/SFU dynamic-source gate, verify that HIGH recovers materially, P2P does not regress, and a constrained SFU downlink still selects and decodes LOW. Do not restore global `motion`, add more simulcast encodings, return to single HIGH, or add a Screener layer selector.
|
||||
2. **Finish real-network route acceptance.** Repeat the controlled active-SFU recovery transaction on exact production. Then measure standard ICE/STUN direct paths, peer relay, SFU ingress/subscription, relay-ingress repair with subtree retention, disconnect/capacity drain, Pause/Resume, screen-audio continuity and real-game A/V sync across representative IPv4/IPv6 NATs and mobile networks. Include Host and Viewer TUN/VPN cases and an all-UDP-blocked case to distinguish an application ICE-option defect from operating-system interception and establish bounded explicit failure. Do not add another watchdog, port prediction, NAT classification, TCP probes, guessed candidates, route scores, or quality-driven reparenting.
|
||||
1. **Close the Viewer lifecycle/native-control release.** Integrate the staged
|
||||
`fix/viewer-page-lifecycle` candidate from the accepted truth checkpoint,
|
||||
run the full repository check, review the exact diff, merge through PR,
|
||||
deploy one immutable application artifact, and perform scoped production
|
||||
postflight. Verify native Viewer controls plus current-route P2P and SFU
|
||||
reconnect without adding route reselection.
|
||||
2. **Finish real-network route and media acceptance.** Exercise direct peer,
|
||||
Browser relay, SFU, relay-ingress recovery with subtree retention,
|
||||
disconnect/capacity drain, Pause/Resume, source replacement, screen-audio
|
||||
continuity, and real-game A/V sync across representative IPv4/IPv6,
|
||||
Wi-Fi/cellular, and Host/Viewer TUN/VPN cases. Include an all-UDP-blocked case
|
||||
and weak-network audio after SFU RED was disabled. Every exhausted path must
|
||||
end in bounded wait/failure without TURN, TCP probing, NAT classification,
|
||||
guessed candidates, or another watchdog.
|
||||
|
||||
## Accepted Later Roadmap
|
||||
|
||||
These items are real product work, ordered after the current physical-media, real-network, and product-decision gates; they are not permission to resume an old branch wholesale.
|
||||
1. **Browser VP8 hardware evidence.** Measure supported Chrome/Edge platforms
|
||||
under real game load using actual codec, encoder implementation, encode time,
|
||||
frame rate, and CPU/GPU attribution. Do not add another codec, custom encoder,
|
||||
GPU selector, or Native helper without a new decision.
|
||||
2. **Mobile Viewer lifecycle.** Run Android Chrome and iOS Safari matrices for
|
||||
autoplay gesture, foreground/background audio, foreground video recovery,
|
||||
lock/page reclamation, rotation, Wi-Fi/cellular migration, and assigned-relay
|
||||
survival or controller recovery. Web does not promise background video or
|
||||
relay execution after OS suspension.
|
||||
3. **Quality-based topology research.** After the real-network baseline, decide
|
||||
whether measured user-visible quality justifies any active parent-selection
|
||||
mechanism. Current evidence cannot compare an active route with an
|
||||
unconnected alternative, so no quality score, all-pairs probing, periodic
|
||||
rebalancing, relay abdication threshold, or active parent switch is accepted.
|
||||
Prefer mature algorithms and one general model if this boundary is reopened.
|
||||
4. **Platform output only when real.** Revisit AirPlay/Cast only when a target
|
||||
browser and physical receiver prove the live `MediaStream` contract. System
|
||||
mirroring remains external.
|
||||
5. **Public-server one-click package.** After functional and real-network work,
|
||||
package the exact application, STUN/SFU, reverse proxy, secrets, and health
|
||||
checks for a user-owned public server. Do not call a partial installer ready.
|
||||
6. **Fully local one-click package.** Package Windows/macOS/Linux Host capture,
|
||||
application server, and local state without requiring source or Node. Report
|
||||
public-origin, TLS, gateway, NAT, and firewall limits honestly.
|
||||
7. **Native Host and shared encode, Windows first.** Productize only after real
|
||||
capture, hardware-only encode, audio, identity, RTP/RTCP feedback, resource,
|
||||
packaging, and licensing gates pass. Browser Host/relay keeps standard
|
||||
per-`RTCPeerConnection` encoding.
|
||||
8. **Whole-product UI and bilingual decision.** Once media and route behavior
|
||||
stabilizes, review copy, responsive hierarchy, visual consistency, restrained
|
||||
motion, bundle/rendering cost, and Chinese/English scope once as a whole.
|
||||
Ordinary screen-specific edits do not create parallel documentation.
|
||||
9. **Repository simplification audit.** After the Browser, route, room, and
|
||||
physical-media checkpoints settle, inventory components, configuration,
|
||||
migrations, timers, compatibility paths, tests, and truth duplication using
|
||||
the [maintenance review](./maintenance.md#机制减负审查). Begin read-only and do
|
||||
not turn it into a broad refactor.
|
||||
|
||||
1. **Browser VP8 hardware-acceleration evidence.** After the current physical codec gate, measure supported Chrome/Edge platform combinations under real game load using actual negotiated codec, encoder implementation, power-efficient status, encode time, frame rate, and CPU/GPU attribution. Capability advertisement alone is not proof. Do not add another codec, a custom encoder, GPU-selection workaround, codec ladder, or Native helper without a new accepted decision.
|
||||
2. **Mobile Viewer lifecycle.** Run Android Chrome and iOS Safari device matrices for autoplay gesture, foreground/background audio, foreground video recovery, lock/page reclamation, rotation, Wi-Fi/cellular migration, and an assigned relay's outbound-media survival or controller failover across those transitions. Preserve the same media element and current reconnect path; do not promise background video composition, background relay continuity, or unsupported lock-screen behavior.
|
||||
3. **Platform output only when real.** Revisit AirPlay/Cast only when a target browser and physical receiver prove the live `MediaStream` contract. System mirroring remains external; no placeholder button, custom receiver, or server transcode is scheduled.
|
||||
4. **Public-server one-click deployment package.** After every current functional and real-network item is settled, package the exact application, STUN/SFU, reverse proxy, secrets, and health checks for a user-owned public server. Require real domain/TLS/firewall/capacity inputs and recovery scoped to any infrastructure or irreversible state the installer actually changes; routine application-only immutable updates do not create or maintain a full configuration backup. Never ship credentials or call a partial deployment ready.
|
||||
5. **Fully local one-click Host-server package.** Package Windows/macOS/Linux Host capture, the exact application server and local state without requiring source, Node or any external Screener/STUN/SFU/tunnel/relay service. Preserve usable direct P2P and use the completed NAT/network evidence to maximize reachability, while reporting public-origin, gateway, NAT and firewall limits truthfully; loopback/LAN success is not universal Internet success.
|
||||
6. **Native Host capture backend.** At lowest priority, productize a proven Windows native capture path before expanding capture/audio backends to macOS and Linux. Its capture, hardware encode, audio, identity and lifecycle evidence becomes an input to later packages; a source checkout, helper executable, or evaluation ZIP is not a distributable product.
|
||||
7. **Native shared encode, Windows first.** At lowest priority, prove one hardware encoder output can serve two independent Viewer transports and the Host-SFU publication while each connection retains its own RTP/RTCP, pacing, encryption, feedback, and upload accounting. Productize only after actual access-unit identity, Viewer2/FIFO, heterogeneous feedback, game-resource, browser/hardware, A/V, packaging, and licensing gates pass. Web Host/relay remains standard per-`RTCPeerConnection` encoding because no portable shared-encoder contract exists.
|
||||
8. **Whole-product UI polish and bilingual-scope decision.** After current functional, physical-media, and real-network work stabilizes, review the product UI once as a whole: simplify and align copy, refine responsive hierarchy and visual consistency, add restrained motion, and measure rendering and bundle cost. Connection presentation uses the two user-facing media labels `P2P` and `SFU` without naming SFU as fallback. A completed reconnect must clear stale “正在重连” stage copy, while retained notices gain an explicit close action. The compact Host roster shows each Viewer's IP for duplicate-name disambiguation without waiting for expanded details; its exact source and data boundary remain part of that task. Decide Chinese/English scope, default/fallback and switching persistence before introducing one shared text catalog; do not add duplicate screen-specific translations or let presentation redefine room and route semantics.
|
||||
9. **Repository simplification audit.** After the current Browser, physical-media, route, and room checkpoints settle, inventory components, configuration, migrations, gates, timers, compatibility paths, tests, and truth-document duplication using the [maintenance review](./maintenance.md#机制减负审查). Begin read-only: identify the behavior or decision each surface preserves and compare that value with its whole-system cost. Any accepted feature tradeoff updates its single owning contract before complete unused surfaces are removed. This later audit must not become a broad refactor or block current product work.
|
||||
## Product Decision Needed
|
||||
|
||||
## Product Decisions Needed
|
||||
|
||||
| Deployed or retained item | Decision |
|
||||
| Item | Decision |
|
||||
| --- | --- |
|
||||
| Project license and closed-commercial boundary | Decide the repository and distribution license before any public release or package distribution. GPL/AGPL implementations remain research-only until then. |
|
||||
| Repository and distribution license | Decide before public release or package distribution. GPL/AGPL implementations remain research-only until then. |
|
||||
|
||||
These decisions are not permission to continue old branches, and current deployment remains unchanged until a scoped decision is made.
|
||||
## Evidence Boundaries
|
||||
|
||||
## Evidence And Later Work
|
||||
|
||||
- The desktop Host background/minimized report is deferred after a bounded current-Chrome screening found no immediate Host-page lifecycle drop and the old `contentHint = "motion"` release remained a plausible confound. Reopen it only with a reproducible current-production real-game case and synchronized capture/send/receive/decode plus CPU/GPU evidence. This does not close or advance the separate Mobile Viewer lifecycle item, whose background playback and assigned-relay survival depend on mobile browser and OS lifecycle behavior.
|
||||
- Open PR #192 measures synthetic flash/tone timing; it is not an A/V correction feature.
|
||||
- The Native PR stack #16/#18/#22/#23/#25/#28 and related fanout branches are one research program, not six product TODOs.
|
||||
- Cap3, resource, route-recovery, Viewer-MBB, signaling-blackhole, Host-generation, SFU-shaping, NAT, mobile, and endurance artifacts remain bounded evidence. Merge or retain them only when a current decision consumes them. Raw-IP ranking, port prediction, NAT classification, AirPlay/Cast receivers, and mobile background guarantees are not implied by the current implementation batch.
|
||||
- Real heterogeneous-network SFU, mobile lifecycle, audio/A-V device, and endurance/resource acceptance remain owned by the corresponding Now or Later evidence gate.
|
||||
- Desktop Host background/minimized capture remains diagnostic until a current-
|
||||
production real-game reproduction correlates capture, outbound, inbound,
|
||||
decode, and CPU/GPU. Viewer resume correction is not a keepalive guarantee.
|
||||
- Synthetic and loopback tests validate invariants, not target-network quality,
|
||||
latency, capacity, mobile lifecycle, or endurance.
|
||||
- Open Native and packaging branches are retained evidence/candidates, not
|
||||
current releases.
|
||||
|
||||
## Candidate Handling
|
||||
|
||||
- Preserve the dirty `fix/configurable-relay-cap` worktree only for user-change audit. Current `main` already implements the accepted uniform `1/2/3` endpoint cap; do not merge the worktree's old route implementation or treat it as unfinished cap work.
|
||||
- Preserve dirty or unique worktrees and open stacked branches until reviewed. Never resolve their conflicts by importing old truth into `main`.
|
||||
- Sample worktree, branch, PR, artifact, and reparse-point state immediately before cleanup; do not maintain a permanent workspace inventory here.
|
||||
- Preserve dirty or unique worktrees and open stacks until reviewed. Never
|
||||
import old truth wholesale.
|
||||
- `fix/media-followup` is clean and tree-equivalent to current `main`; it may be
|
||||
removed only after merged-head, reference, and reparse checks.
|
||||
- Sample worktree and branch state immediately before cleanup rather than
|
||||
maintaining a permanent inventory here.
|
||||
+91
-43
@@ -1,56 +1,104 @@
|
||||
# Current Verification Status
|
||||
# Verification Status
|
||||
|
||||
Last updated: 2026-08-25
|
||||
|
||||
## Purpose
|
||||
This ledger records evidence that still changes how current source or production
|
||||
may be interpreted. Git and pull requests own routine completed checks.
|
||||
|
||||
This is the demand-loaded ledger for cross-cutting validation evidence and open proof boundaries that do not fit one requirement, ADR, research note, or deployment guide. It is current state, not a release diary: replace superseded facts in place, move durable specialist conclusions to their owning documents, and let Git history retain completed timelines.
|
||||
## Current Production Identity
|
||||
|
||||
Capacity and routing entries below describe the exact tested source or release only. Product direction and executable work are owned by [project memory](./project-memory.md) and [the TODO ledger](./todo.md).
|
||||
- Production runs exact application/runtime revision
|
||||
`af348ee1d508a3af02b18a7f46c461953798e19d`, release `af348ee`, wire
|
||||
`screener-v12`, from `/opt/screener/releases/af348ee`.
|
||||
- Runtime tar SHA-256:
|
||||
`4922cc31d7caaad7c5412b1f2fe1d73f788d16d92386efad6eac358a97b8a78e`.
|
||||
The 39-file manifest SHA-256 is
|
||||
`5b397809b664b9a55885ac603df615b537bf51276e7274416820870956242d7d`.
|
||||
- Public Browser asset `assets/index-BvuZY6Hc.js` is 488473 bytes with SHA-256
|
||||
`e8fc0500496ee101031a9cfebfcc22d3b69e66d2cac5e875448b56620dbb21c5`.
|
||||
Public `/healthz` returns 200. Release postflight found Screener, LiveKit,
|
||||
coturn, and nginx active with zero restarts.
|
||||
|
||||
## Deployment Evidence
|
||||
## Completed Product Evidence
|
||||
|
||||
- Production runs exact deployed application/runtime revision `be53c4d5794d38d5d406c876309958d7117ee601`, release `be53c4d`, wire `screener-v12`, from `/opt/screener/releases/be53c4d`; that revision is integrated in canonical `main`. The immutable runtime tar SHA-256 is `5c174c30ffacd98738a364b825b915b84432823c261272684937361be2451144`; its 39-file manifest SHA-256 is `c34dba43a25c6b8447e1153a5be3191bf82cff94117ed31edc325d3f7cf8541f`.
|
||||
- Postflight found `screener`, LiveKit, coturn, and nginx active with zero restarts and no warning-level Screener/LiveKit journal entries. Local/public health and the served main Browser asset matched the release; `assets/index-BKEVI7Ux.js` has SHA-256 `22114fc8a26b3f03f37973555433d1e73d52d1c7b76fa3c7cc597500823bd62d`, while the prior main asset returns 404.
|
||||
- Production uses endpoint capacity `2`, room admission `20`, random free four-digit memory rooms, `ROOM_LEASE_SECONDS=86400`, a dedicated LiveKit instance with `room.auto_create: false` and `max_participants: 21`, and global SFU admission `1` ingress / `20` egress. Room authority is process-memory-only. Coturn is `stun-only`, `no-tcp`, and `no-tls`; only UDP 3478 listens, and nft exposes only UDP 3478/7882 for Screener media.
|
||||
- The production service orders Screener after LiveKit and boundedly waits for its control listener. Media listeners remain STUN-only UDP 3478 and LiveKit UDP 7882 with no 5349, coturn TCP/TLS, relay range, ICE/TCP, or media TCP listener. The stable `inet bonfire_filter` table stayed unchanged through cutover at SHA-256 `afaf9d066e9f292d8bd19032b38c0978ccf60dd01c2fe7f210968ff751f90534`; the whole ruleset is not used as an identity because fail2ban owns a dynamic address set.
|
||||
- This application-only postflight proves archive identity, atomic application/assets, process health, and listener/firewall continuity. It did not re-exercise access, stale-wire rejection, direct, browser-relay, SFU, audio, codec, mobile, quality, or endurance behavior; those retained results remain revision-bounded below. Routine application-only releases do not maintain a full rollback/configuration backup; any future infrastructure or irreversible-state change requires recovery scoped to what it changes.
|
||||
- Strict `screener-v12` rejects stale Browser/executable wires before room
|
||||
authority. Process-memory rooms, 24-hour dormant leases, restart loss,
|
||||
room-lived Viewer grants, `open | private` code entry, invitation-only private
|
||||
rooms, password entry, grant rotate/revoke, and `ROOM_NOT_FOUND` have passed
|
||||
source and production gates.
|
||||
- Endpoint capacity `1/2/3`, default `2`, one active upstream, acyclicity,
|
||||
source reachability, deterministic parent order, exact candidate identity,
|
||||
first-decoded-frame commit, strictly newer rollback, bounded-gap resource
|
||||
accounting, SFU admission, and delete-plus-absence release have focused unit
|
||||
and controller coverage.
|
||||
- Controlled Chrome routing proved direct prepare failure, rollback, SFU
|
||||
prepare, exact first-frame ready, SFU commit, and continuing UDP media without
|
||||
external ICE servers on Browser LiveKit PCs. This does not replace public
|
||||
heterogeneous-network recovery evidence.
|
||||
- Chrome direct and Browser-relay VP8 measurements reached about 60 fps after
|
||||
stock bandwidth-estimation warm-up. Pinned LiveKit client `2.22.0` and server
|
||||
`1.13.5` separately proved native lower/higher screen-share representation
|
||||
selection with Dynacast and server send-side BWE.
|
||||
- Direct, Browser-relay, and SFU paths advanced video plus screen audio at
|
||||
64/128/256 kbps ceilings and after source replacement. The old constant
|
||||
active-audio `1 kbps` display is closed. Exact SFU weak-network behavior after
|
||||
RED was disabled remains open.
|
||||
- Browser SFU publisher/subscriber PCs use empty external ICE-server lists while
|
||||
ordinary peers retain deployment STUN. The reported Host/Viewer TUN conflict
|
||||
was closed by that isolation without adding TURN or TCP media.
|
||||
|
||||
## Retained Functional Evidence
|
||||
## Current Source Evidence
|
||||
|
||||
- Exact deployed source `be53c4d` passed repository hygiene, TypeScript typecheck, all 613 Web tests in 44 files, client and server production builds, two independent reviews, pinned LiveKit encoding calculation, and production artifact/service postflight. Exact predecessor `b68c471` separately passed the 261-test SFU admission gate and explicit-Chrome access/privacy gate. Retained Chrome 20-Viewer and responsive visual checks apply only to their recorded predecessor revisions.
|
||||
- Chrome 151 on exact production `1d87615` used real dynamic-tab `getDisplayMedia()` at 1822x1080 and made only the ordinary STUN peer attempt unusable so the existing route operation reached its SFU suffix. Publisher and subscriber LiveKit PCs retained empty external ICE-server lists and selected UDP. The publisher exposed one video sender encoding and one outbound video stats object, both without RID; over six seconds it encoded 180 frames while the subscriber decoded and rendered 182 frames, with bytes advancing at both ends. This closes the single-representation implementation/deployment gate without claiming public SFU 60 fps, real-game quality, heterogeneous-network behavior, or endurance.
|
||||
- Chrome 151 on exact production `1d87615` used one continuously changing synthetic Browser stream while test-side policy made only ordinary STUN peer PCs unusable and left LiveKit unchanged. The Viewer observed direct prepare revision `1`, explicit rollback active revision `2`, SFU prepare revision `3`, exact-revision first-frame ready, then SFU active revision `3`; no ready was sent for the failed direct candidate. The final Host snapshot had no operation, parent/final route `sfu`, rejection `none`, and first-frame/final timing `10703/10703` ms within the unchanged 20-second total deadline. Host and Viewer LiveKit PCs had zero external ICE URLs, selected UDP, and advanced `58/57` frames plus bytes over the next two seconds. This closes the exact deployed controlled candidate rollback/SFU commit transaction, not physical capture, heterogeneous-network reachability, all-UDP-blocked failure, quality, or endurance.
|
||||
- Chrome 151 on exact production `1d87615` formed `Host -> SFU -> relay -> child` plus a separate direct Viewer, passed Pause/Resume, and then locally closed the relay's subscriber PC. After 45,234 ms the relay remained assigned to SFU and sent no `route-media-unavailable`, `refresh-sfu`, or `route-failed`; its downstream child independently reported its own decoded stall and recovered through the other Viewer. Receiver-stats calls had stopped about 28.9 seconds before the final sample. Source inspection confirms that no video track stops SFU sampling while an empty merged report or failed stats read produces no deadline sample. This bounded fault injection exposes an active-cadence implementation gap; local `RTCPeerConnection.close()` does not reproduce physical UDP loss or prove LiveKit's real reconnect path, so relay-ingress recovery remains open.
|
||||
- Chrome 151 on exact production `bf32859` captured a continuously changing tab with generated audio through real `getDisplayMedia()`. Two Host-direct children plus one browser-relay child and one forced-SFU Viewer all advanced decoded audio energy and video frames at 64/128/256 kbps sender ceilings and after live source replacement. Ordinary paths selected direct UDP; the SFU publisher/subscriber selected LiveKit UDP with empty external ICE-server lists. The UI never reported the faulty constant `1 kbps` with active audio. Direct/browser-relay actual traffic tracked each selected ceiling; SFU 64/128 traffic was higher because the accepted LiveKit default retains RED, so sender ceiling/readback and wire traffic remain distinct. After RTCP mapping became available, direct/browser-relay audio-minus-video playout timestamps ranged from -101 to +4 ms; Chrome exposed no corresponding value on the SFU subscription, so real-game SFU A/V synchronization remains open rather than inferred.
|
||||
- Chrome 151 on exact production `bf32859` used real `getDisplayMedia()`, a continuously changing source, separate Browser processes, VP8/libvpx, and no video hint. Direct stabilized near 59.5 encoded/received/decoded/rendered fps at 1904x928; a Browser relay kept every stage near 59.4--59.7 fps, adapting its outbound to 1428x696 under bandwidth pressure. Both converged after about 25--30 seconds of stock BWE ramp-up. On the exact public SFU path, the then-current `q,h` publication held `h` near 9.2 fps/1.37 Mbps; making `q` inactive from the first sender-parameter application raised the same 1904x928 `h` to about 23.3 fps/3.32 Mbps with source/capture near 60, 4.46 ms encode time per frame, zero loss/retransmission, and about 4.81 Mbps available outgoing bitrate. That gate closed an intrinsic VP8/no-hint direct/relay shortfall and rejected always-active `LOW`; it did not prove a deployed single-representation SFU path, public SFU 60 fps, or real-game performance.
|
||||
- Exact Browser SFU ICE-server-isolation implementation `ae09c760adec76fd26da611d4928486d105c6d3b` passed repository hygiene, TypeScript typecheck, all 605 Web tests in 45 files, client and server production builds, three independent code reviews, focused 95-test publisher/subscriber and ordinary-peer isolation coverage, deployment postflight, and owner physical SFU fallback media proof. This establishes the reported production connection fix without turning candidate type/count into an invariant; full route/network acceptance remains open.
|
||||
- A fresh canonical `main@c4962f5` background gate passed repository hygiene, all 605 Web tests, both production builds, the 254-test SFU admission suite, and the Chrome 151 20-Viewer 720p30/30-second local route gate. All 20 Viewers advanced decoded frames, Host and relays stayed within endpoint cap `2`, no orphan SFU publication appeared, and all four current-child timing distributions contained 20 values. The browser gate uses deterministic canvas and does not start LiveKit, so it proves local direct/browser-relay control and media behavior only; it is not physical capture or SFU evidence.
|
||||
- Exact v10 runtime source `fdd5a4a529ff297f41c05ea3388bf484d76afe8f` passed repository hygiene, TypeScript typecheck, client and server production builds, all 610 Web tests in 45 files, the 257-test SFU admission gate, the access/privacy Browser gate, and the production-dependency audit with zero vulnerabilities. Its one-Host/20-Viewer Chrome 151 direct-loopback report passed all seven assertions: all Viewers decoded, the selected VP8 settings reached every participant, Host and relay media edges stayed within cap `2`, no orphan SFU publication appeared, and all four current-child timing distributions contained 20 values. The report SHA-256 is `802CBB7632E9AEA3C39FE831A7F7880D24140B2E83942CE0DFDA05DE4B408EED`. This validates the v10 source boundary, not H.264 quality, hardware attribution, physical capture, public-network behavior, or SFU packet flow.
|
||||
- Chrome 151 synthetic `1/3/5/8` and 720p30 runs preserved bounded fanout and decoding. A `9461e20`-era three-Viewer rerun kept Host2/Browser Viewer1, decoded every Viewer, and recovered a hard first-level relay departure in 5,380 ms; it did not exercise quality-triggered MBB. Sixteen-Viewer cap2/cap3 loopbacks decoded all Viewers and a ten-second follow-up recorded guarded sender and aggregate Chromium CPU evidence, but resolution/FPS remained low and the ordinary-PC data cannot rank caps, select product policy, or prove sustainable heterogeneous-network quality. See [peer-assisted media](./research/peer-assisted-media.md).
|
||||
- Chrome 151 with local LiveKit 1.13.5/client 2.22.0 proved one SFU/UDP root after `q,f` -> `q,h`: decoded frames advanced 3 -> 23, rendered frames reached 26, Host used one edge, and leaves were clean. This is functional evidence only. See [low-server media routes](./research/low-server-media-routes.md).
|
||||
- Chrome 151 on the current Windows TUN path reproduced intermittent production LiveKit initial ICE before publication: server counters observed both ingress and egress while Browser ICE received no responses. Eight isolated Host attempts using the LiveKit-provided STUN endpoint succeeded `2/8`; explicitly empty Browser SFU `iceServers` succeeded `8/8`, observed a peer-reflexive-to-host UDP pair within 0.5--1.7 seconds, and published 75--89 ms later. This selects Browser SFU ICE-server isolation; the observed candidate type is diagnostic, not a gate, and the run does not close exact deployed Host/Viewer first-frame, mobile, quality, or endurance evidence. See [low-server media routes](./research/low-server-media-routes.md).
|
||||
- Host A+B and authenticated P2P Viewer C evidence is sanitized, generation-bound, diagnostic-only, and closed for stale or ambiguous reports. Production exposes one Host-local current SFU sender snapshot and clears it on publisher identity changes; raw media metadata is not retained. See [realtime quality adaptation](./research/realtime-quality-adaptation.md).
|
||||
- Fresh Chrome 151 tab-capture probes identified two separate Browser boundaries. Removing video `contentHint = "motion"` changed VP8 from 14.93 fps at 428x208 to 29.73 fps at 1904x928 under the same 5 Mbps balanced setup. No-hint AMD H.264 remained about 14--16 fps with hundreds of outer rate-limiter drops, while the same MFT reached 29.93 fps under diagnostic Desktop SW BRC; OpenH264 was slower. The page cannot enable that Chromium field trial or select an adapter/MFT. No-hint VP8 used libvpx software and held about 30 fps in one- and two-sender loopback on the test machine, but the process CPU totals include capture and local decode and are not real-game or weaker-device evidence. This evidence selects fixed VP8 for the Browser contract; current source and production implement that decision, and exact-production gates verify VP8 on direct, Browser-relay, and single-representation SFU paths. See [realtime quality adaptation](./research/realtime-quality-adaptation.md), [Native H.264](./research/native-h264-hardware-decision.md), and [Native shared encode](./research/native-shared-encode-sender.md).
|
||||
- One bounded Chrome 151/Windows 25H2 current-browser screening used real `getDisplayMedia()` window capture, VP8/libvpx with no video hint, a continuously changing WPF source, and one same-machine loopback receiver. Focused, hidden-tab, and minimized 15-second windows measured about 29.4--29.6 capture/encoded/decoded fps and 1.35--1.50 ms encode time per frame; each boundary sample read an enabled, unmuted, live 1186x712 track and each end sample reported `qualityLimitationReason=none`. All rates used monotonic counter deltas over the outbound-RTP RTCStats timestamp interval; source/inbound timestamps and raw stats identities were not retained independently. The sample did not observe an immediate Host-document background drop; it does not cover exact production code, a real game, physical Viewer, CPU/GPU contention, audio, repetition, a hold beyond five minutes, or public routing. See [Browser background capture](./research/browser-background-capture.md).
|
||||
- Native Windows 11 process audio is source-only and default-off. A retained run isolated the target by about 4018x and delivered 495 Opus packets to one Viewer; package download, game sync, Windows 10, and other routes remain open. See [ADR-0008](./adr/0008-window-scoped-audio-capture.md).
|
||||
- `af348ee` fixes Browser VP8 with `contentHint = "motion"`, reapplies the
|
||||
selected video profile after answer negotiation, leaves SFU representation
|
||||
construction to pinned LiveKit, keeps Dynacast/send-side BWE, disables
|
||||
AdaptiveStream, and disables SFU audio RED.
|
||||
- The accepted `fix/viewer-page-lifecycle` candidate has focused coverage for
|
||||
page-resume decoded-stall rebaselining, current-frame presentation authority,
|
||||
and same-parent P2P manual rebuild. TypeScript and the client production build
|
||||
pass. It is not current source or production until its release is integrated.
|
||||
|
||||
## Open Proof Boundaries
|
||||
## Open Physical Gates
|
||||
|
||||
- Routing: exact v10 local direct loopback remains historical evidence for admission-20 initial ingress. Exact v12 production closes one controlled ordinary-direct exhaustion, explicit rollback, and exact SFU first-frame commit transaction; the later subscriber-PC closure injection found the active-SFU cadence gap without proving physical loss recovery. Current source corrects that cadence while physical recovery remains open. Representative real-network direct/peer/SFU behavior, relay-ingress reparent with subtree retention, disconnect/effective-capacity drain, pause/resume, and bounded all-UDP-blocked failure remain open. Silent control partitions may wait 30 to 60 seconds for heartbeat detection before the default five-second grace; a non-paused decoded-frame stall may invalidate only that exact child edge.
|
||||
- Quality: exact-production direct and Browser-relay VP8/no-hint cadence plus the historical single-encoding SFU baseline are closed. ADR-0007's pinned gate separately proved LiveKit-owned selection between full- and half-resolution representations with server send-side BWE, and production now caps the half-actual-resolution lower representation at 30 fps with proportional bitrate. Its mixed-route HIGH recovery, constrained LOW delivery on this release, public SFU 60 fps, real-game quality, heterogeneous devices, and endurance remain open. Startup ramp is stock BWE behavior and does not authorize an SDP or application bitrate workaround.
|
||||
- Host background: deferred until the report reproduces on exact production with a continuously changing real game or replay, a separate direct wired Viewer, synchronized capture/outbound/inbound plus CPU/GPU evidence, and repeated focused/unfocused/hidden/minimized windows. The current-Chrome Browser `getDisplayMedia()` WPF-window screening did not observe an immediate Host-page media drop, and the old harmful `contentHint = "motion"` release remains a plausible confound rather than a proven background cause. Current evidence authorizes no keepalive, Wake Lock, timer, codec switch, route change, or Native fallback.
|
||||
- Relay resources: current source and production expose the authenticated `1/2/3` endpoint-cap boundary, and retained v10 local loopback evidence covers 20 Viewers at cap `2` only for that source. CPU/GPU/upload and heterogeneous-network measurements remain diagnostics, not authority to change the contract.
|
||||
- Room scale: current source and production admit 20 Viewers plus one Host, and the retained v10 local Browser convergence gate covers that bound on direct loopback. A real public-network 1:20 session, quality, and sustained resource matrix remain open.
|
||||
- SFU: current source and production use one half-actual-resolution lower representation capped at 30 fps plus the original representation, Dynacast, and the tracked server send-side-BWE setting without an application layer selector. Both use exact Viewer subscriptions and no fixed root-count or room-wide lease rule. Chrome 151 physically closed the empty-external-ICE-server connection hold and historical single-representation gate, while the separate pinned-stack gate proved native representation selection. The lighter lower representation's production quality effect, real LiveKit lifecycle/JWT replay, resource use, SFU-fed first-child behavior, and broader heterogeneous-network acceptance remain open.
|
||||
- Audio: current v12 source and production select the exact active ordinary Host or SFU publisher audio sender, reset samples across track changes or absence, keep relay children synchronized with persistent-stream audio changes, and reacquire exact SFU publications/senders after reconnect. Chrome 151 on exact production `bf32859` physically closed the reported active-source `1 kbps` observation and all three live ceilings across direct, browser-relay and SFU. Audible game quality, heterogeneous-network changes and real-game A/V synchronization remain part of route acceptance; a muted/idle Opus track may still legitimately remain near 1 kbps.
|
||||
- Observability: current v12 source and production keep local connection details but no Host/Viewer diagnostic downloads or Browser file-export plumbing. The Host-only bounded snapshot stays for automatic acceptance. Chrome 151 supplied selected candidate endpoints and ordinary-route A/V playout timestamps in the current gate, but omitted the SFU A/V timestamp. Target-browser availability and actual values remain open for codec/fmtp and encoder implementation/power efficiency, media-source and encoded FPS, and broader jitter/concealment cases. All retained fields diagnose and have no recovery authority; they do not by themselves prove hardware encode, synchronization quality, or route quality.
|
||||
- Mobile: Android/iOS remain Viewer-only. Autoplay, rotation, iOS lock-screen/page reclamation, background reconnection, and network changes need device evidence. AirPlay/system mirroring is Viewer-local output, not a portable in-app WebRTC output contract.
|
||||
- Native: ADR-0006 has one-Viewer evidence only. Viewer2/FIFO, hardware diversity, endurance, downloaded-package/public-host, game A/V, and browser diversity remain open.
|
||||
- Access and operations: exact predecessor `b68c471` passed the access/privacy gate with explicit Chrome and production cutover/postflight, including stale-v11 rejection before authority, room-lived grant/code orthogonality, private entry with and without a password, rotate/revoke, missing-room typing, and memory-only runtime. Headful same-browser preference replay and the isolated VM/IP clean-port matrix remain open; shared-IP smoke cannot approve them.
|
||||
### Route And Network
|
||||
|
||||
## Evidence Rules
|
||||
- Repeat direct, peer-relay, SFU, relay-ingress recovery with subtree retention,
|
||||
disconnect/capacity drain, and Pause/Resume on representative IPv4/IPv6,
|
||||
Wi-Fi/cellular, and Host/Viewer TUN/VPN networks.
|
||||
- Verify all-UDP-blocked networks reach bounded explicit failure. No TURN,
|
||||
ICE/TCP, media TCP, TLS relay, NAT classification, port prediction, or TCP
|
||||
probe is authorized.
|
||||
- Run the 20-Viewer public-network capacity/resource/endurance gate. Local or
|
||||
synthetic 1:20 control evidence does not establish target-network quality.
|
||||
|
||||
- Target-device and production measurements drive performance decisions. Ordinary-PC synthetic runs prove only correctness or interoperability.
|
||||
- Record expensive evidence with exact source, environment, result, and applicability. When a relevant media path changes, replace or invalidate the corresponding entry; unrelated documentation changes do not require rerunning it.
|
||||
- Keep architecture-splitting candidates out of this ledger unless a proven consumer boundary makes them active work; file length alone is not evidence.
|
||||
### Media Quality
|
||||
|
||||
- Measure real games at 720p30, 1080p30, and 1080p60 across weaker Hosts and
|
||||
actual Browser encoder implementations. Correlate capture, outbound, inbound,
|
||||
decode, CPU/GPU, and A/V timing.
|
||||
- Verify pinned LiveKit default representation behavior on exact production
|
||||
under constrained and unconstrained subscribers without adding an application
|
||||
layer selector.
|
||||
- Verify weak-network screen audio and A/V synchronization after RED was
|
||||
disabled, including concealment/FEC evidence where exposed.
|
||||
|
||||
### Lifecycle
|
||||
|
||||
- Desktop Host background/minimized capture remains unconfirmed. A short
|
||||
current-Chrome window-capture screening did not reproduce an immediate drop;
|
||||
a current-production real-game reproduction must separate Host page, captured
|
||||
surface, capture, encoder, path, and Viewer behavior.
|
||||
- Android Chrome and iOS Safari still need autoplay, background audio,
|
||||
foreground video recovery, page reclamation, rotation, network migration, and
|
||||
assigned-relay survival tests. Web does not promise background video or relay
|
||||
execution after OS suspension.
|
||||
|
||||
## Interpretation Rules
|
||||
|
||||
- Configuration and loopback prove neither target-network quality nor hardware
|
||||
acceleration.
|
||||
- Missing RTCStats fields are unknown, not zero.
|
||||
- Current-path quality is diagnostic and cannot authorize active parent change.
|
||||
- A Browser page lifecycle correction is recovery logic, not keepalive.
|
||||
- Native, packaging, and broad UI work remain outside these gates until their
|
||||
TODO decisions are opened.
|
||||
+12
-12
@@ -2,7 +2,7 @@
|
||||
|
||||
## 2026-08-18 14:45
|
||||
|
||||
状态：production 运行精确 application/runtime revision `1d8761528d0dba43fb6d818df3934483ba2f5340`、release `1d87615` 和严格 `screener-v12`；canonical `main` 含相同 application/runtime tree 与当前 truth checkpoint。源码与生产已实现 22 字符 room-lived Viewer grant、`ROOM_NOT_FOUND`、产品诊断下载移除、完整短 URL 邀请 UI、收敛的房间密码控件、exact-current audio stats 与音轨跨 relay/SFU reconnect 连续性，并固定 Browser VP8，video 不设置 `contentHint`、audio 保留 `music`，且无 codec UI/state/wire 或分享生命周期分支；Pause/Resume 继续由当前 Host session 和 `shareGeneration` 约束。Browser SFU 发布为单个无 RID 的 `HIGH` encoding，不启用 simulcast 或 Dynacast；Chrome 151 已在该精确 production 上验证发布端与订阅端经 LiveKit UDP 持续推进帧。
|
||||
状态：production 与 canonical `main` 运行精确 application/runtime revision `af348ee1d508a3af02b18a7f46c461953798e19d`、release `af348ee` 和严格 `screener-v12`。Browser 视频固定 VP8 并使用 `contentHint = "motion"`，音频使用 `music`；codec UI/state/wire、自定义 SFU layer list 与应用媒体质量 controller 均不存在。SFU 由 pinned LiveKit 默认 representations、Dynacast、server send-side BWE 和 disabled AdaptiveStream 拥有质量适配；screen audio 使用 64/128/256 kbps ceiling、stereo、DTX off 与 RED off。当前 Viewer 页面生命周期/原生控件候选已接受但尚未集成或部署，精确执行状态由 `docs/status.md` 持有。
|
||||
|
||||
## 目标与边界
|
||||
|
||||
@@ -158,7 +158,7 @@ accepted 的唯一 wire 是 `screener-v12`，不做版本协商、旧 parser、
|
||||
| 1080p30 | 1920x1080、30 fps | 5 Mbps/peer |
|
||||
| 720p30 | 1280x720、30 fps | 3 Mbps/peer |
|
||||
|
||||
浏览器可返回更低设置，界面以 `track.getSettings()` 显示实际捕获结果。推荐区始终只有表中的三档并默认选择中档 `1080p30`；高级分辨率额外提供 `480p`（854x480）、720p、1080p 与 1440p，FPS 继续独立接受 15 至 60 整数，码率继续独立接受 2 至 12 Mbps，因此不建立第四个推荐档或 `480p` preset ID。视频 track 不赋任何 `contentHint`，让 display capture 保留浏览器的 screen 分类；返回的音频 track 使用 `contentHint = "music"`。三档推荐质量与折叠高级区初始值统一采用 `degradationPreference = "balanced"`，并允许显式选择 clarity-first 或 fluid。“分享高级设置”不提供 codec 选择，并提供离散的 `64 kbps 省流 | 128 kbps 音乐默认 | 256 kbps 极高音质` 屏幕音频 sender ceiling。音频 ceiling 可在直播中串行更新。这些设置只是 ceiling 和浏览器偏好，不保证实际分辨率、帧率、码率或声道。
|
||||
浏览器可返回更低设置，界面以 `track.getSettings()` 显示实际捕获结果。推荐区始终只有表中的三档并默认选择中档 `1080p30`；高级分辨率额外提供 `480p`（854x480）、720p、1080p 与 1440p，FPS 继续独立接受 15 至 60 整数，码率继续独立接受 2 至 12 Mbps，因此不建立第四个推荐档或 `480p` preset ID。视频 track 使用标准 `contentHint = "motion"` 表达游戏动态意图；返回的音频 track 使用 `contentHint = "music"`。三档推荐质量与折叠高级区初始值统一采用 `degradationPreference = "balanced"`，并允许显式选择 clarity-first 或 fluid。“分享高级设置”不提供 codec 选择，并提供离散的 `64 kbps 省流 | 128 kbps 音乐默认 | 256 kbps 极高音质` 屏幕音频 sender ceiling。音频 ceiling 可在直播中串行更新。这些设置只是 ceiling 和浏览器偏好，不保证实际分辨率、帧率、码率或声道。
|
||||
|
||||
每次 `setParameters()` 都必须从当次 `getParameters()` 派生更新，然后读回 requested/applied `maxBitrate`、`maxFramerate`、`scaleResolutionDownBy` 和 degradation preference。浏览器负责具体降级，实际结果由 stats 观测；拒绝或浏览器改写必须进入可见状态，不能只写 `console.warn`。
|
||||
|
||||
@@ -176,19 +176,19 @@ Web display-audio 请求显式关闭 `echoCancellation`、`noiseSuppression`、`
|
||||
|
||||
Host 本地预览直接绑定现有 capture stream，不创建 viewer、`RTCPeerConnection` 或额外媒体边，也不产生 SFU/应用服务器出口。预览仍可能有本地合成、缩放和 GPU 显示成本；在同一动态场景对预览开/关采集 Host CPU、GPU、encode time 和实际 FPS。后台性能诊断必须把 Host 页面前台/遮挡/最小化、被捕获 source 前台/遮挡/最小化以及 preview 播放/暂停分成独立变量，并关联 document visibility/focus、track mute/readyState 与 capture/outbound/inbound stats。页面 timer、伪活跃和 Screen Wake Lock 都不构成屏幕捕获保活方案；若标准浏览器无法维持目标行为，应记录平台能力边界而不是制造第二套 lifecycle。只有稳定收益且捕获、编码和远端画面不变时，才把直播后的默认动态预览改为静止缩略图或隐藏并允许按需展开；隐藏预览绝不停止 capture track 或 sender encode。
|
||||
|
||||
codec 与 content-hint 依据由研究文档持有。`motion` 会把 Chromium/libwebrtc 的 display capture 从 screen 改成 realtime，并在同一 VP8 条件下造成明显降帧和降分辨率，因此产品不设置视频 hint。无 hint VP8 在当前受控证据中恢复接近 30 fps、完整分辨率和正常码控，具备 Browser 基线覆盖；它因此成为唯一 Browser codec。普通 AMD H.264 MFT 的浏览器码控仍丢帧，而诊断性 Desktop SW BRC 可恢复约 30 fps，说明问题不等于硬件编码器本身的吞吐上限；网页没有启用该浏览器进程开关或选择具体 GPU/MFT 的 API。实际 codec 与 encoder 字段只用于诊断，不触发 codec 切换。VP8 hardware acceleration 需要后续按目标浏览器和真实游戏负载单独取证；本设计不假定其存在，也不增加替代 codec 或自定义编码路径。
|
||||
codec 与 content-hint 依据由研究文档持有。`motion` 会改变 Chromium/libwebrtc 的适配取舍并可能空间降级以保留游戏动态；它是标准内容意图，不是分辨率或帧率保证。产品不再用 unset hint 固定 screen-classified 行为，也不围绕某次测量实现应用缩放器。普通 AMD H.264 MFT 的浏览器码控仍丢帧，而诊断性 Desktop SW BRC 可恢复约 30 fps，说明问题不等于硬件编码器本身的吞吐上限；网页没有启用该浏览器进程开关或选择具体 GPU/MFT 的 API。实际 codec 与 encoder 字段只用于诊断，不触发 codec 切换。VP8 hardware acceleration 需要后续按目标浏览器和真实游戏负载单独取证；本设计不假定其存在，也不增加替代 codec 或自定义编码路径。
|
||||
|
||||
`track.getSettings()` 中存在的 EC/NS/AGC/voice-isolation、`channelCount` 和 `sampleRate` 只能作为当前捕获观测；缺失字段保持未知，Opus 协商仍是独立事实。每个 Web Host 或浏览器 relay 的实际音频 sender 在首次绑定和换源后通过标准 `RTCRtpSender.setParameters()` 请求并读回当前 64/128/256 kbps ceiling；这只是上限，浏览器 WebRTC 拥塞控制仍可降低实际流量，不新增应用层音频降级器。128 kbps 保持默认，因为 RFC 7587 将 20 ms full-band stereo music 的建议区间列为 64--128 kbps，pinned LiveKit 2.22.0 也把 128 kbps 命名为 `musicHighQualityStereo`；256 kbps 是用户显式选择的较高上限，不是质量保证。Peer answer 声明 `stereo=1;maxaveragebitrate=256000`，让同一接收协商覆盖所有三档，实际 sender ceiling 仍由所选档位约束；普通 PC synthetic 不用于标定参数、听感或性能。
|
||||
|
||||
Web peer 的最小 stereo 契约只改 Viewer 生成的 answer。实现把 MIT `sdp-transform` 2.15.0 与 `@types/sdp-transform` 声明为 root direct dependencies，不依赖 LiveKit 的 transitive install，也不用字符串或正则拼接 SDP。Helper 仅在解析出唯一非 rejected audio media、通过结构化 `rtp` entry 找到唯一 Opus payload 时，才在该 payload 的 fmtp 幂等 upsert `stereo=1` 与 `maxaveragebitrate=256000`；随后 `ViewerPeer` 把同一个结果交给 `setLocalDescription()` 和 signaling。首次连接、ICE restart、同 parent rebuild与浏览器 relay child 都走该边界。空 SDP、无或歧义 Opus、多个有效 audio、重复目标参数、畸形 fmtp、parse/write 异常都按对象 identity 返回浏览器原 answer，保住仍可继续的 mono/video edge，不发送半修改 SDP。RFC 7587 说明这两个值都是可选的单向 receiver parameters；`maxaveragebitrate` 是 maximum 而非最低保证，实际发送仍可随拥塞下降。当前 libwebrtc 把缺省 full-band stereo codec bitrate 配成 64 kbps，而 `RTCRtpSender.maxBitrate` 只限制上界；本接收上限允许三档，实际发送预算仍由 64/128/256 kbps sender ceiling 与拥塞共同决定。`sprop-stereo` 只是 sender likelihood hint，`usedtx` 缺省已是 0，本 slice 不增加二者，也不改浏览器协商的 `useinbandfec`。
|
||||
|
||||
SFU publisher 将 64 kbps 映射到 `AudioPresets.musicStereo`、128 kbps 映射到 `musicHighQualityStereo`，256 kbps 使用显式 `maxBitrate`；三者都保留 `forceStereo: true` 与 `dtx: false`。LiveKit 2.22.0 先合并 room publish defaults，其中 `red: true`；本实现不传 `red` 覆盖值，保留既有 RED request。RED 与 Opus in-band FEC 是不同修复机制，保留前者不声称实际启用、替代或等价于后者。直播中只改变档位时，不重新协商 Opus 或 republish；服务端沿现有 quality wire 保存 last-wins desired profile，各客户端 sender-mutation queue 从 fresh parameters 更新当前 P2P、relay 和 SFU audio sender并立即 readback，过期 completion 不覆盖新值，未来 sender 读取最新 desired profile。应用失败保留该 sender 的旧 applied 值和媒体并显示告警；无远端 applied 回执时 Host 不宣称全房原子收敛。Pinned LiveKit 2.22.0 的当前 Chrome/Edge screen-audio baseline 可直接更新现存 audio sender；其他浏览器以实际 readback 为准。Viewer 继续在同一个持久 `MediaStream`/媒体元素中增删 screen video/audio track；不新建 audio element、Web Audio mix/resample graph、第二 audio representation 或独立音频时钟。默认每条 outbound edge 的 audio RTP payload ceiling 为 128 kbps，显式最高档为 256 kbps；非服务器端点容量为 C 时最多 C × 128/C × 256 kbps（不含 RTP/SRTP/UDP/IP overhead）。stereo 仍可能增加编码与网络成本，且浏览器 relay 不保证跨 PeerConnection shared encode。未来 microphone voice 若进入范围，另建独立 source/track 与 AEC、降噪、gain、DTX 设计，不能作用于电影/游戏屏幕音频。详细证据与 Oopz 未知边界见 `docs/research/browser-screen-audio-quality.md`。
|
||||
SFU publisher 将 64 kbps 映射到 `AudioPresets.musicStereo`、128 kbps 映射到 `musicHighQualityStereo`，256 kbps 使用显式 `maxBitrate`；三者都使用 `forceStereo: true`、`dtx: false` 与 `red: false`。RED 与 Opus in-band FEC 是不同修复机制；关闭 RED 降低冗余流量并接受其弱网恢复退让，不声称禁用或替代 Opus FEC。直播中只改变档位时，不重新协商 Opus 或 republish；服务端沿现有 quality wire 保存 last-wins desired profile，各客户端 sender-mutation queue 从 fresh parameters 更新当前 P2P、relay 和 SFU audio sender并立即 readback，过期 completion 不覆盖新值，未来 sender 读取最新 desired profile。应用失败保留该 sender 的旧 applied 值和媒体并显示告警；无远端 applied 回执时 Host 不宣称全房原子收敛。Viewer 继续在同一个持久 `MediaStream`/媒体元素中增删 screen video/audio track；不新建 audio element、Web Audio mix/resample graph、第二 audio representation 或独立音频时钟。详细证据见 `docs/research/browser-screen-audio-quality.md`。
|
||||
|
||||
Viewer 本地连接详情只从当前 route 的唯一 inbound screen video/audio stats 读取同步证据：`audio estimatedPlayoutTimestamp - video estimatedPlayoutTimestamp` 表示当前发送端时间线差；音视频 jitter-buffer delay 取相邻窗口累计 delay/count 的比值；音频 concealment 取相邻窗口 concealed/total sample 与 event 增量。每个媒体种类用 stats id、SSRC 和 track identifier 共同守住区间身份，换流或累计计数回退时先重新基线，字段缺失或分母为零时保持未知。该证据不进入 signaling、质量上报或持久化，也不设置 `jitterBufferTarget`、Web Audio 或任何媒体控制参数；真实同步仍由同一持久 `MediaStream`/媒体元素和 WebRTC playout 负责。
|
||||
|
||||
`frameRate` 约束和 sender 的 `maxFramerate` 都是目标或上限；浏览器仍控制实际捕获、重复帧处理、编码和拥塞降级。静态画面可能由浏览器减少实际编码数据，但标准没有保证每个实现以同一种方式节省采集或 GPU 开销。本 PoC 不添加画面差分、定时改约束或自定义动态 FPS 状态机，先用 `framesEncoded`、实际发送码率、`totalEncodeTime`、质量受限原因和主机资源占用测量真实问题。
|
||||
|
||||
质量自适应采用 ADR-0007 的路径隔离框架能力，而不是全房间按最差链路降档。Direct/peer 保留每条 PeerConnection 的 stock WebRTC GCC。Browser Host 的唯一 SFU publication 不手写层列表，由锁定的 LiveKit client 以半分辨率和原始分辨率两路 VP8 screen-share encoding 提供空间表示。LiveKit Dynacast 按全房最高需求停用其上的发布层，无订阅时停用全部层；启用 send-side BWE 的 server stream allocator 独立选择每个 subscriber 的实际下行层，订阅端最高质量请求只是上限。Screener 不实现质量评分、带宽估计、周期调档或应用媒体层 selector。当前任一 Viewer 都可能承担 peer children，因此 subscriber 统一保持 `adaptiveStream: false`，relay ingress 不跟随本地 DOM 可见性。Pinned gate 已证明受限下游原生选择半分辨率、非受限下游取得原始分辨率、手动及元素驱动的同连接恢复和 Dynacast 启停；也证明最高需求会让两路同时编码。双编码是支持受限下游的固定成本，不是回退到单路的理由。当前源码已完成迁移，生产仍是单路 encoding。
|
||||
质量自适应采用 ADR-0007 的框架所有权，而不是全房间按最差链路降档。Direct/peer 保留每条 PeerConnection 的 stock WebRTC GCC。Browser Host 的唯一 SFU publication 不手写层列表，由锁定的 LiveKit client 默认行为拥有 VP8 screen-share representations、source replacement 与 republish。LiveKit Dynacast 和启用 send-side BWE 的 server stream allocator 独立处理聚合需求与每个 subscriber 的实际下行层，订阅端最高质量请求只是上限。Screener 不实现质量评分、带宽估计、周期调档或应用媒体层 selector。当前任一 Viewer 都可能承担 peer children，因此 subscriber 统一保持 `adaptiveStream: false`，relay ingress 不跟随本地 DOM 可见性。Pinned gate 证明了 LiveKit 原生空间选择和 Dynacast 能力，但具体 representation 参数仍属于 pinned component，而不是 Screener 产品常量。
|
||||
|
||||
质量证据只用于诊断，不形成全房分数或 parent authority。路由只消费 exact child edge 的 `failed/closed` 和非暂停 decoded-frame stall。prepare/commit/abort 绑定 authenticated sessions、share generation、base/pending revision 与 candidate connection；candidate child 的第一帧新 decoded video 是唯一应用层 media-ready，ICE connected 不能单独提交。
|
||||
|
||||
@@ -206,7 +206,7 @@ Viewer 本地连接详情只从当前 route 的唯一 inbound screen video/audio
|
||||
|
||||
Viewer 的已鉴权质量证据只描述当前路径，不能命令 WebRTC/LiveKit 改层、改变 Host publication 或改变拓扑。UA、平台和设备型号不参与质量或 relay capacity 决策。
|
||||
|
||||
SVC 的标准能力门禁已经静态关闭为 `no-go-web-svc-cross-path-hardware-contract`。WebRTC-SVC 只配置单个 outgoing sender，direct/peer receiver 没有对应的按层选择 API，独立 PeerConnection 之间也没有 shared-encode 契约；requested/applied codec/`scalabilityMode`、Media Capabilities `powerEfficient` 与 RTCStats encoder 字段不能证明编码器一定是硬件。Pinned LiveKit 虽能在 SFU 内为每个 subscriber 设置最大空间/时间层请求，但实际选择依 codec/selector 而异：server 1.13.5 只为 VP8 安装 temporal selector，当前 H.264 `q,h` simulcast 只有空间层选择。Client 2.22.0 还会把 screen-share SVC 强制成三层 temporal-only `L1T3`，无法提供当前 Browser 合同要求的 LOW/HIGH 两种空间表示；这种 SFU 选层也不延伸到 direct/peer descendants。未来只能由能明确锁定硬编、最多两层、跨 direct/peer/SFU 复用同一输出且通过游戏性能/延迟门槛的 native sender 以新决策重开。普通非分层编码包不能只靠转发生成另一档画质，实现路径差异只能用第二表示、合格的 scalable layers 或 relay/SFU 转码。
|
||||
SVC 的标准能力门禁已经静态关闭为 `no-go-web-svc-cross-path-hardware-contract`。WebRTC-SVC 只配置单个 outgoing sender，direct/peer receiver 没有对应的按层选择 API，独立 PeerConnection 之间也没有 shared-encode 契约；requested/applied codec/`scalabilityMode`、Media Capabilities `powerEfficient` 与 RTCStats encoder 字段不能证明编码器一定是硬件。Pinned LiveKit 可在 SFU 内为每个 subscriber 设置最大空间/时间层请求，但实际选择依 codec、SDK 默认和 server selector 而异，且不延伸到 direct/peer descendants。未来只能由能明确锁定硬编、跨 direct/peer/SFU 复用合格输出且通过游戏性能/延迟门槛的 native sender 以新决策重开。普通非分层编码包不能只靠转发生成另一档画质，实现路径差异只能由届时接受的 scalable representation 或转码模型拥有。
|
||||
|
||||
质量控制不放宽稳态 outbound media-copy 预算：所有非服务器端点继续受统一静态 capacity 约束；native relay 只转发选定表示的编码包，不解码重编码。后续双树或条带化必须另行验收，不能绕过当前 graph、identity 和 capacity invariant。
|
||||
|
||||
@@ -295,15 +295,15 @@ Host 与 relay Viewer 复用同一个纯 presentation reducer 保存远端 `View
|
||||
首屏就是工具，不做营销页。视觉采用中性深灰工作台，绿色表示 direct、琥珀色表示 relay/警告、红色表示失败，避免单一蓝紫色主题；浏览器 favicon 复用页头 `MonitorUp` 的深绿底、绿色显示器和上行箭头语言。
|
||||
|
||||
- 访问入口：未鉴权根路径、`/join` 和合法 code-only `/r/{code}` 显示“站点访问”与“站点口令”；malformed `/r/...` 直接显示房间号必须是 `1000..9999` 四位数字，其他未知路径只显示“无法访问”。有效 `/r/{code}#v={grant}` 直接进入；其余 Viewer 通过 site access 后以 code-only 方式进入。未分配、已回收或已过期的 well-formed code 显示“房间不存在或已过期”，不展开或提供密码输入；现存房间的 `ROOM_ACCESS_DENIED` 保留“输入房间密码”和使用 Host 邀请的动作。已有有效 site-access cookie 不重复显示整站验证；站点口令和房间密码不共享输入值或授权状态。
|
||||
- 分享者根页面：顶部显示“{分享者显示名} 的屏幕”和房间状态，并在标题旁显示可单独复制的紧凑四位房间号；Viewer 同一位置显示权威 Host 名称，不再在标题下重复。主区域保持稳定的 16:9 舞台。未发布、已停止或可恢复错误状态时，舞台内以同等尺寸和权重显示“开始分享”和“加入房间”，中间只用“或”与分隔线表达发送/接收；固定 action row 只能包含这两个按钮及分隔，disclosure 是紧随其后的 sibling，不得参与 action-row track sizing。前者直接开始现有分享流程，后者在当前舞台内展开共享的房间码表单，提交后才导航到 `/r/{code}`。按钮同步 `aria-expanded`/`aria-controls`，展开后输入获得焦点，DOM 与焦点顺序保持触发器后接表单；窄屏允许 disclosure 自己换行，不改变两个 action 的矩形或产生横向滚动。不增加首页、向导或拓扑选择。直播时舞台显示本地预览及独立全屏图标，底部保留停止、同步暂停音视频和质量档位。邀请控制使用紧凑 copy-first 布局：包含 22 字符 grant 的完整短 URL 保持可见，复制是主动作；更新和撤销使用有 tooltip/accessible name 的 familiar icons，房间号复制、`公开 | 私密` code-entry segmented control 与可选密码编辑保持职责分离。页面失焦仅暂停本地 video element，并在舞台提示“本地预览已暂停，分享仍在继续”；媒体暂停则禁用 capture stream 轨道，两者都保留现有连接。
|
||||
- 分享者根页面：顶部显示“{分享者显示名} 的屏幕”和房间状态，并在标题旁显示可单独复制的紧凑四位房间号；Viewer 同一位置显示权威 Host 名称，不再在标题下重复。主区域保持稳定的 16:9 舞台。未发布、已停止或可恢复错误状态时，舞台内以同等尺寸和权重显示“开始分享”和“加入房间”，中间只用“或”与分隔线表达发送/接收；固定 action row 只能包含这两个按钮及分隔，disclosure 是紧随其后的 sibling，不得参与 action-row track sizing。前者直接开始现有分享流程，后者在当前舞台内展开共享的房间码表单，提交后才导航到 `/r/{code}`。按钮同步 `aria-expanded`/`aria-controls`，展开后输入获得焦点，DOM 与焦点顺序保持触发器后接表单；窄屏允许 disclosure 自己换行，不改变两个 action 的矩形或产生横向滚动。不增加首页、向导或拓扑选择。直播时舞台只显示无媒体 controls 的本地预览，权威停止、同步暂停音视频、切换来源和质量档位保持为明确的页面动作。邀请控制使用紧凑 copy-first 布局：包含 22 字符 grant 的完整短 URL 保持可见，复制是主动作；更新和撤销使用有 tooltip/accessible name 的 familiar icons，房间号复制、`公开 | 私密` code-entry segmented control 与可选密码编辑保持职责分离。页面失焦仅暂停本地 video element，并在舞台提示“本地预览已暂停，分享仍在继续”；媒体暂停则禁用 capture stream 轨道，两者都保留现有连接。
|
||||
- `/join`：保留为可直接访问的回退入口，并复用同一个四位房间码表单；一个纯 route parser 返回 `host | join | viewer | malformed-room | unknown`，只把精确 `/r/{1000..9999}` 当成 Viewer route。输入保留用户原文，`inputMode="numeric"` 只提示软键盘，提交时由共享 schema 拒绝而不是 `trim`、删除非数字或接受 Unicode 数字。通过 site access 后输入合法 code 并进入 `/r/{code}`，后续行为由 code-only admission 决定。
|
||||
- `/r/{code}` 观看者：首次解析合法 fragment grant 后立即清理地址栏并直达；仅有 code 时先通过 site access，再按 `open/private` policy 进入。私密房间统一返回通用拒绝并允许尝试密码，不披露房间是否配置了密码。鉴权后主区域最大化远端视频，标题旁显示可复制的紧凑房间号，顶部保留连接状态，并提供本地音量、静音、全屏和重试控制；Host 主动暂停时复用视频舞台遮罩显示“分享者已暂停”，恢复后立即移除。视频舞台用单一派生 presentation reducer 合成 access、Host presence、route prepare/active、peer/SFU connection、remote track、首个 composited frame、autoplay rejection 和 bounded failure。支持时用 `requestVideoFrameCallback()` 确认第一帧已提交，旧浏览器回退到当前代 decoded-frame delta 与 video readiness；`playing`、track object 或 ICE `connected` 单独都不足以结束占位层。连接阶段占用固定布局且不显示虚假百分比，播放按钮只处理实际 autoplay rejection，失败重试只调用现有有界 controller 入口。
|
||||
- `/r/{code}` 观看者：首次解析合法 fragment grant 后立即清理地址栏并直达；仅有 code 时先通过 site access，再按 `open/private` policy 进入。私密房间统一返回通用拒绝并允许尝试密码，不披露房间是否配置了密码。鉴权后主区域最大化远端视频，标题旁显示可复制的紧凑房间号，顶部保留连接状态；同一个 video element 使用浏览器原生 controls 处理本地播放、音量和全屏，页面只保留“重新连接媒体”。Host 主动暂停时复用视频舞台遮罩显示“分享者已暂停”，恢复后立即移除。视频舞台用单一派生 presentation reducer 合成 access、Host presence、route prepare/active、peer/SFU connection、remote track、首个 composited frame、autoplay rejection 和 bounded failure。支持时用 `requestVideoFrameCallback()` 确认第一帧已提交，旧浏览器回退到当前代 decoded-frame delta 与 video readiness；`playing`、track object 或 ICE `connected` 单独都不足以结束占位层。当前代 composited frame 清除同 revision 的旧 connecting/reconnecting 展示；页面恢复先 rebaseline decoded stall。手动重连只重建同一 P2P parent 或同一 SFU subscription，不主动重选线路。
|
||||
- 首帧观察绑定当前 assignment/connection/media-binding generation 以及适用的 publication generation；`srcObject` 或 route identity 变化立即使旧回调和旧 decoded baseline 失效，迟到回调不能把新路线标成已播放。
|
||||
- 移动端：视频保持稳定宽高比，控制与状态移到视频下方，任何文字和按钮不得溢出或遮挡。
|
||||
|
||||
两端默认显示紧凑在线 roster；Viewer 端复用同一权威 participant snapshot，只列出当前 Viewer（含本机）并在新 snapshot 到达时整体替换，停止分享仅移除已离线 Host，鉴权失效、房间关闭或页面卸载则清空。Host 与 Viewer 的本机显示名在界面中统一标为“昵称”，默认只读，铅笔按钮紧贴昵称并只切换当前控件的输入、保存和取消，唯一名称仅显示名称，重名时才追加 room-scoped `peerId` 短后缀。一个不持久化的“显示连接详情”复选框按页展开媒体方式、传输协议、候选路径、浏览器实际暴露的本地/远端候选地址、codec、RTT、RTP 丢包率、编解码耗时及逐边统计；地址与端口只从当前页面 `RTCPeerConnection.getStats()` 精确关联的 selected pair 读取，浏览器隐藏或缺失任一字段就不显示，且不进入质量证据、信令、服务端、日志或持久化。界面不提供诊断文件下载。真实连接失败、音频缺失、持续画质受限和需要用户动作的告警始终可见。显示名/roster 不进入 access credential 或服务器持久状态。
|
||||
|
||||
用户可见的非直播状态使用稳定、真实的阶段词：分享端使用“已停止分享”；观看端使用“等待开始分享”“正在分配线路”“正在建立 P2P”“正在连接备用线路”“正在接收画面”“需要点击播放”“正在恢复连接”或明确失败。阶段只描述当前用户可感知动作，不暴露房间寿命、媒体代次、parent identity 或内部迁移步骤。`room-closed` 显示“房间已关闭”或“房间已过期”，不得误写成“分享已结束”。服务端 raw `message` 永不直接渲染；单一纯 reducer 按下表优先级把 typed current-generation fact 派生为文案、遮罩、保留画面和动作，较旧 revision/generation 输入直接丢弃：
|
||||
用户可见的非直播状态使用稳定、真实的阶段词：分享端使用“已停止分享”；观看端使用“等待开始分享”“正在分配线路”“正在建立 P2P”“正在连接 SFU”“正在接收画面”“播放已暂停”“正在恢复连接”或明确失败。阶段只描述当前用户可感知动作，不暴露房间寿命、媒体代次、parent identity 或内部迁移步骤。`room-closed` 显示“房间已关闭”或“房间已过期”，不得误写成“分享已结束”。服务端 raw `message` 永不直接渲染；单一纯 reducer 按下表优先级把 typed current-generation fact 派生为文案、遮罩、保留画面和动作，较旧 revision/generation 输入直接丢弃：
|
||||
|
||||
| 优先级 | 权威输入 | 展示与动作 |
|
||||
| ---: | --- | --- |
|
||||
@@ -318,7 +318,7 @@ Token-free SFU standby prewarm 保持可选且不创建 participant、publicatio
|
||||
|
||||
按钮使用 `lucide-react` 图标并提供可访问名称和 tooltip。动态状态区域固定最小尺寸，避免连接状态变化导致布局跳动。在 1440、1024、780、520、390、375 与 320 CSS px 下，closed/open/validation-error 三种 disclosure 状态的两个首屏 action 都须同排等宽、触控高度至少 44 px且前后 `getBoundingClientRect()` 尺寸不变；输入获得焦点，文字不溢出，详情不遮挡视频且 `scrollWidth <= clientWidth`。
|
||||
|
||||
观看端使用原生 `range` 展示 `0%` 至 `100%`，并把归一化值直接写入当前 `<video>.volume`；它不进入 WebRTC sender、信令、房间状态或 `localStorage`。React 页面状态跨 `srcObject` 替换、换父和 ICE 恢复保留；页面刷新后恢复默认值。静音按钮只切换本地 `muted`，并保留上次非零音量；拖到零自动静音，从零调高或直接取消静音时恢复可听状态。该控件不作为音频码率、声道、采样率或来源隔离设置。
|
||||
观看端不维护第二套音量、静音、播放或全屏状态；浏览器原生 video controls 是这些本地动作的唯一 owner。它们不进入 WebRTC sender、信令、房间状态或应用持久化，也不作为音频码率、声道、采样率或来源隔离设置。
|
||||
|
||||
## 安全与故障边界
|
||||
|
||||
@@ -339,7 +339,7 @@ Token-free SFU standby prewarm 保持可选且不创建 participant、publicatio
|
||||
- 进程重启后 room state 全空、旧 Host/grant/password fail closed、释放号码复用时旧 grant 仍失败；`ROOM_LEASE_SECONDS` 的默认/边界与 `MAX_ROOMS <= 9000` 校验，已删除的 `ROOM_DATABASE_PATH`/`ROOM_TTL_SECONDS` fail-fast；
|
||||
- STUN UDP URL 严格校验、生产 STUN 必需、LiveKit tuple 完整性/TTL/secret 独立性、当前 route/session/revision/connection generation guard，以及 ordinary peer、未授权房间和 Native 的 STUN-only isolation；LiveKit `sfu-config` 保持独立边界；
|
||||
- Accepted v12 contract 使用单一 `screener-v12` 信令 literal、严格 JSON schema、方向与房间授权校验、payload 上限和 client-mismatch 刷新终态；分享状态由 `shareGeneration`-fenced paused boolean 表达，missing room 使用 `ROOM_NOT_FOUND`，现存房间准入失败使用 `ROOM_ACCESS_DENIED`，Viewer route failure 为 typed terminal，Host-only bounded route snapshot 保留给自动验收但不进入产品 UI。`qualityProfileIdSchema` 保持三档，`qualityResolutionSchema` 包含 `480p`，`DEFAULT_QUALITY_SETTINGS` 为 1080p30，quality schema 严格拒绝 codec 字段；
|
||||
- 质量协议的 host 授权、严格字段与数值范围、last-wins、当前/未来 relay、已配置 SFU 和普通 P2P wire 兼容；sender 的四字段 requested/applied 读回、拒绝与持续受限提示；screen audio SDP helper 覆盖 CRLF、existing/missing fmtp、unknown key preservation、幂等、payload number、rejected/non-audio，以及 empty/no-Opus/ambiguous/parser 异常原对象 fallback，`ViewerPeer` 覆盖 initial/restart/rebuild 时同一结果的 local/signaling 一致性，SFU publisher 精确断言 high-quality stereo/forceStereo/DTX 且不覆盖 RED；
|
||||
- 质量协议的 host 授权、严格字段与数值范围、last-wins、当前/未来 relay、已配置 SFU 和普通 P2P wire 兼容；sender 的四字段 requested/applied 读回、拒绝与持续受限提示；screen audio SDP helper 覆盖 CRLF、existing/missing fmtp、unknown key preservation、幂等、payload number、rejected/non-audio，以及 empty/no-Opus/ambiguous/parser 异常原对象 fallback，`ViewerPeer` 覆盖 initial/restart/rebuild 时同一结果的 local/signaling 一致性，SFU publisher 精确断言 high-quality stereo/forceStereo/DTX/RED-off；
|
||||
- HTTP/WS 集成：nginx per-source site-access `limit_req` 与错误不泄露 secret，应用不建立 IP limiter 状态；Viewer 可在 Origin/容量门下 upgrade；Host 首条鉴权同时要求 upgrade-captured site access 与 Host token，Viewer 要么持 exact-room grant，要么先有 site access 再按 `open/private`。覆盖原子创建 profile、同浏览器重建偏好、fragment/sessionStorage/replaceState、无 site access 的 code-only 拒绝、unknown/reclaimed/expired room 使用 `ROOM_NOT_FOUND` 且不显示密码、私密无密码与 wrong-or-absent-password/full code-only 使用 `ROOM_ACCESS_DENIED`、grant 与 Host 的独立错误、raw grant/密码不进入 URL/日志/服务端持久状态，以及 rotate/revoke 后 grace/route 清理、旧 Viewer/上游媒体边关闭和 reconnect 拒绝；
|
||||
- 客户端策略与 stats 解析：会话替换不重连，ordinary ICE 与 LiveKit protocol 分离；
|
||||
- SFU client isolation：`SfuPublisher` 与 `SfuSubscriber` 的 `Room.connect` mock 分别精确断言 `autoSubscribe: false` 与 `rtcConfig: { iceServers: [] }`，ordinary `HostPeer`/`ViewerPeer` 回归继续断言部署 STUN，防止空列表遗漏 Viewer 或扩散到 peer edge；
|
||||
@@ -362,7 +362,7 @@ Token-free SFU standby prewarm 保持可选且不创建 participant、publicatio
|
||||
|
||||
## 当前交付边界
|
||||
|
||||
- Production 使用严格单一 `screener-v12` wire 和精确 deployed application/runtime revision `1d8761528d0dba43fb6d818df3934483ba2f5340`、release `1d87615`；current source 含待部署的新 application 与 LiveKit template。两者均已实现短 room-lived grant、明确 missing-room 结果、产品诊断下载移除、完整短 URL 邀请 UI、exact-current audio stats 与音轨跨 relay/SFU reconnect 连续性，固定 Browser VP8 且 video 不设置 `contentHint`，codec selector、quality-state/wire 字段与分享生命周期 codec 分支不存在。Current source 的 Browser SFU 使用 pinned-default 半分辨率加原始分辨率 simulcast 与 Dynacast；production 仍只发布一个 encoding。
|
||||
- Production 与 canonical `main` 使用严格单一 `screener-v12` wire 和精确 deployed application/runtime revision `af348ee1d508a3af02b18a7f46c461953798e19d`、release `af348ee`。两者均实现短 room-lived grant、明确 missing-room 结果、产品诊断下载移除、完整短 URL 邀请 UI、exact-current audio stats 与音轨跨 relay/SFU reconnect 连续性，固定 Browser VP8、video `contentHint = "motion"`、audio `music`，且不存在 codec selector、quality-state/wire 字段、自定义 SFU layer list或分享生命周期 codec 分支。Browser SFU 使用 pinned LiveKit 默认 representations、Dynacast、server send-side BWE、disabled AdaptiveStream 与 RED off。
|
||||
- Pause/Resume 由当前 Host session 和 `shareGeneration` 约束。Host 后台性能和 VP8 hardware acceleration 分别按受控根因证据诊断；标准 ICE/SFU 按真实异构网络验收。
|
||||
- 当前发布面只包含 Web Host、Viewer 和 browser relay。Native sender、capture helper、shared-encode executable 与两个分发包仍由各自后续决策和物理证据驱动，不属于当前 Browser release。
|
||||
|
||||
|
||||
+14
-14
@@ -19,16 +19,16 @@ SFU 的 publication、subscription、server-resource accounting、临时 overlap
|
||||
- 首屏双入口：分享入口通过站点访问后，主工作区必须同等突出“开始分享”和“加入房间”两个主要动作，并用清楚的视觉分隔表达发送与接收；持合法房间 grant 的观看者不需要知道站点访问密码。“加入房间”在当前舞台内就地展开房间码输入并提交，不先跳转到独立页面；直接访问 `/join` 仍提供同一输入能力。不增加营销首页、模式向导或拓扑选择。浏览器标签页使用与现有 `MonitorUp` 品牌语言一致的 favicon。
|
||||
- 简单房间：分享者从当前空闲池随机获得一个 `1000` 至 `9999` 的四位房间码，房间码只负责定位，不是秘密或永久身份。Host 与 Viewer 标题附近都紧凑、显眼地显示房间号并提供只复制房间号的动作；每个新房默认生成含独立 Viewer grant 的完整邀请链接，Host 的主分享动作复制该链接。房间号入口只有公开和私密两种状态；新房默认公开，允许已通过站点访问的 Viewer 直接用房间号加入且不设房间密码。私密且未设置密码时只有有效邀请可加入；设置密码后才额外允许房间号加密码加入。邀请链接与房间号准入相互独立。所有入口只授予该房间 Viewer，不能建房、发布、冒充 Host 或跨房。
|
||||
- 免安装观看：桌面和手机观看者可直接通过浏览器链接进入、授权并播放，不依赖与分享端相同的桌面客户端。
|
||||
- 屏幕采集：分享者可主动选择显示器或应用窗口，开始、停止、临时同步暂停音视频和切换分享来源；暂停保留现有连接并禁用当前 capture stream 的全部媒体轨道。服务端按当前分享代次同步这一主动暂停状态，观看端在视频舞台显示“分享者已暂停”，不得从黑帧、track mute 或网络状态推断暂停；晚加入和信令重连使用权威快照，恢复、停止、新分享或会话替换清除旧提示。Host 本地预览必须直接使用 capture stream，支持独立全屏；页面失焦或隐藏时只暂停本地 video element，并明确提示“本地预览已暂停，分享仍在继续”，恢复焦点后继续预览，不得停止 capture、编码或上行。预览不得建立 viewer/媒体边或消耗 SFU、应用服务器 media egress；预览仍可能产生本地合成、缩放和 GPU 显示成本，不能把隐藏预览称为停止采集或编码。
|
||||
- 屏幕采集：分享者可主动选择显示器或应用窗口，开始、停止、临时同步暂停音视频和切换分享来源；暂停保留现有连接并禁用当前 capture stream 的全部媒体轨道。服务端按当前分享代次同步这一主动暂停状态，观看端在视频舞台显示“分享者已暂停”，不得从黑帧、track mute 或网络状态推断暂停；晚加入和信令重连使用权威快照，恢复、停止、新分享或会话替换清除旧提示。Host 本地预览必须直接使用 capture stream；页面失焦或隐藏时只暂停本地 video element，并明确提示“本地预览已暂停，分享仍在继续”，恢复焦点后继续预览，不得停止 capture、编码或上行。预览不得建立 viewer/媒体边或消耗 SFU、应用服务器 media egress；预览仍可能产生本地合成、缩放和 GPU 显示成本，不能把隐藏预览称为停止采集或编码。
|
||||
- 游戏声音：Windows Chrome/Edge 的初次分享和切换来源都默认请求可用的标签页、窗口或系统音频；浏览器、系统、所选来源或用户没有返回音轨时必须明确提示，而不是静默无声或阻止 video-only 分享。
|
||||
- 观看端本地音量：播放控制栏提供 `0%` 至 `100%` 音量条和静音按钮，只改变当前浏览器的媒体元素，不修改 Host 音轨、编码、房间状态或其他观看者。切流、换父和 ICE 恢复保留当前页面内的值；刷新后不持久化。静音保留上次非零音量，拖动到非零值自动取消静音。
|
||||
- 观看端本地播放：Viewer 使用同一个带浏览器原生 controls 的 video element 承载播放/暂停、音量、静音和全屏；这些动作只影响当前浏览器，不修改 Host 音轨、编码、房间状态、路由或其他观看者。Host 本地预览不提供媒体 controls，权威暂停仍由明确的“暂停分享”动作拥有。
|
||||
- 本地昵称与房间成员：用户可设置一个只保存在当前浏览器同源 `localStorage` 的显示名，界面统一称为“昵称”；默认以只读文本显示，铅笔编辑按钮紧贴昵称，点击后才进入编辑。Viewer 顶部标题直接显示“{Host 名称} 的屏幕”，不得在下方重复一行 Host 名称。信令只在当前在线 socket 会话内保存和广播 NFC 规范化、最多 24 个 Unicode code points、拒绝控制/换行/双向覆盖符与孤立 surrogate 的纯文本标签，不写服务端持久存储或日志。空值使用通用角色 fallback。Web Host 与 Viewer 都必须看到全部当前在线 Viewer 的权威总数与 roster，不能拿最多两条 Host 媒体直连数冒充在线人数；presence 更新也不得创建媒体连接。展示身份默认只显示名称；仅当房内出现同名时追加当前房间 `peerId` 的至少六字符短后缀，并只在同名项后缀碰撞时有界延长。显示名、后缀和 `clientId` 都不是账号、授权身份或全局 UUID，重复名称允许且不得参与授权、路由或质量决策。Web Host 与 Viewer 通过同一 opt-in participant roster 看到 Host 名称；Native/executable sender 不在当前发布面，也不接收 Web roster。
|
||||
- 一对多观看：每个房间允许一名 Host 和最多 `20` 名 authenticated Viewer；房间 admission 与端点下游容量、SFU admission 分别配置，超额加入必须明确拒绝。观看人数不从 endpoint cap 推导，20-Viewer smoke 只证明控制面与资源上界，不是任意网络质量声明。
|
||||
- 端点发送容量：所有非服务器端点使用同一个服务端权威稳态 outbound media-copy capacity，由唯一部署项 `ENDPOINT_MEDIA_COPY_CAPACITY` 配置，默认 `2` 且只接受 `1`、`2` 或 `3`；不读取旧键或保留第二套 tier。peer child 或 Host 的唯一 SFU publication 各消耗一份，上游接收不消耗该预算。客户端只能声明不高于部署值的可用容量，角色、浏览器、UA、设备类型和页面可见性不产生不同上限。服务端在每次 `authenticated` 权威快照中返回最终 `endpointMediaCopyCapacity`；普通非 peer-assisted 模式也只授权稳定加入顺序中的至多 `C` 个 Host child，超额 Viewer 保持入房等待并在槽位释放后提升，信令不能绕过该集合。Browser Host 与 Viewer 严格校验该值并用它约束稳态 assignment；并行产生媒体的 candidate 必须持有唯一 `TransitionOverlapSlot`，active+pending 物理 copy 只可达到 `min(C + 1, 3)` 并在 commit/abort 后释放。`C=3` 满载时必须先收到 active bounded-gap retirement 再 prepare，绝不产生第四份 copy。SFU subscriber egress 由独立 server admission 记账。当前不兼容房间合同以单一 release 原子升级；缺失/非法 cap、旧 wire 与 executable sender都在取得room/route authority前fail closed。
|
||||
- P2P 优先：每条普通逻辑媒体边先尝试 direct/peer UDP，服务器不得默认中转可直连媒体。更多观看者优先沿满足统一端点容量的无环 peer 图分布；每名 Viewer 同时至多一个 active upstream，失败时只影响需要重选的 child。任何 source-reachable endpoint 在 effective capacity 有空位时都可作为 parent candidate；Peer-fed 与 SFU-fed endpoint 使用同一个 provisional-child transaction，并在 candidate child 解码出第一帧新画面后独立提交该 edge。Host 已满且首次需要 SFU 时，同一 reconcile 将 affected Host direct child（若无则稳定顺序中最新的 connected Host direct child）作为本次 child operation，创建唯一 Host publication 并把该 child 变为首个 SFU subscriber，subtree 不动；publication 取代释放的 Host peer slot，下一次扫描再处理原受限 Viewer，不建立 bootstrap 状态机。
|
||||
- NAT 与传输边界：STUN 必须可用，所有 ordinary peer `RTCPeerConnection` 只接收 STUN ICE servers。Browser LiveKit publisher/subscriber PC 必须显式使用空 `iceServers`，保留 LiveKit signaling 交付的授权 SFU UDP candidates，并由标准 ICE 选择 nominated non-relay pair；不得把 ordinary-peer STUN、LiveKit join response 中的 STUN 或默认公共 STUN混入该 PC，也不得把某一种 candidate type 或数量写成成功条件。部署 STUN 仍供 ordinary peers 和 LiveKit server-side 公网地址发现使用。浏览器 ICE connectivity checks 是 direct UDP 可达性的权威；Screener 不合成 remote candidate、不预测端口、不分类 NAT，也不使用 TCP 探测推断 UDP 媒体可达性。所有恢复和候选授权必须绑定 authenticated room/endpoint session、share generation、base/pending route revision 与 candidate connection identity；成功即停止，失败、超时或绑定失效即撤销，direct 阶段耗尽后在同一 operation deadline 内进入 SFU/UDP，全部可用路径耗尽后明确等待或失败。已接受的完整顺序只有保持或尝试 direct/STUN peers，再复用或建立 direct-ingress Host publication/SFU subscription。Screener 不配置 TURN、ICE/TCP、media TCP 或 TLS-relayed media；当前 LiveKit 也保持 UDP-only。未来若真实严格防火墙矩阵证明需要另一种传输，必须把它作为 LiveKit 内部能力重新决策，不能增加第三种应用 route candidate。LiveKit join JWT 仅按协议进入不得记录的 WebSocket request target；HTTPS/WSS 继续独立使用 TLS/TCP。
|
||||
- 质量控制：推荐档位始终只有 1080p60（8 Mbps）、1080p30（5 Mbps）、720p30（3 Mbps）三档，默认选择中档 `1080p30`。高级设置的分辨率选项增加 `480p`（854x480），其 FPS 与码率仍由高级控件独立选择；不增加 `480p` 推荐档、preset ID 或固定的 480p FPS/码率组合。三档与高级设置初始值统一采用 `balanced`（平衡），`maintain-resolution`（清晰优先）、`maintain-framerate`（流畅优先）保留为显式选择。Browser 视频固定使用 VP8；界面、质量状态和 wire 不提供 codec 选择或切换。分享期间可直接换视频档位和 64/128/256 kbps 音频 sender ceiling，不重新选择来源或重建健康连接。Peer-assisted relay 与已配置的 SFU 必须接收同一个严格校验的房间级权威设置，覆盖当前和以后建立的 sender；普通 P2P 信令结构保持不变。任何设置都只是浏览器上限和偏好，具体降级由浏览器决定；必须读回实际 sender 参数并通过 stats 观测结果、显示参数拒绝、改写和持续受限原因。Host 资源不足时由 Host 主动选择较低的现有档位，不得以删除 SFU LOW 表示来静默牺牲受限下游。不做自定义动态 FPS、综合评分或周期性全房间调档。
|
||||
- 路径隔离表示：direct/peer 继续使用每条 PeerConnection 独立的 stock WebRTC GCC，不把 Viewer 反馈合成全房目标。Browser Host 的唯一 SFU publication 将层配置留空，由锁定的 LiveKit client 提供半分辨率和原始分辨率两路 VP8 screen-share encoding；LiveKit Dynacast 按全房最高需求停用其上的发布层，无订阅时停用全部层，最高需求则保留两路编码；启用 send-side BWE 的 server stream allocator 为每个 subscriber 选择实际下行层。Screener 不实现质量评分、带宽估计、周期调档或应用媒体层 selector。所有当前 Viewer 都可能承担 peer children，因此 subscriber 保持 `adaptiveStream: false`，不得让本地元素尺寸或可见性控制 relay ingress。表示选择不改变拓扑、端点容量或路由失效权威。ADR-0007 的 pinned-stack 物理门禁已经通过；当前源码已实现该组合，生产仍是待迁移的单路 encoding。
|
||||
- 质量控制：推荐档位始终只有 1080p60（8 Mbps）、1080p30（5 Mbps）、720p30（3 Mbps）三档，默认选择中档 `1080p30`。高级设置的分辨率选项增加 `480p`（854x480），其 FPS 与码率仍由高级控件独立选择；不增加 `480p` 推荐档、preset ID 或固定的 480p FPS/码率组合。三档与高级设置初始值统一采用 `balanced`（平衡），`maintain-resolution`（清晰优先）、`maintain-framerate`（流畅优先）保留为显式选择。Browser 视频固定使用 VP8；界面、质量状态和 wire 不提供 codec 选择或切换。分享期间可直接换视频档位和 64/128/256 kbps 音频 sender ceiling，不重新选择来源或重建健康连接。Peer-assisted relay 与已配置的 SFU 必须接收同一个严格校验的房间级权威设置，覆盖当前和以后建立的 sender；普通 P2P 信令结构保持不变。任何设置都只是浏览器上限和偏好，具体降级由 WebRTC/LiveKit 决定；必须读回实际 sender 参数并通过 stats 观测结果、显示参数拒绝、改写和持续受限原因。Host 资源不足时由 Host 主动选择较低的现有档位。Screener 不设置自定义 SFU layer list、动态 FPS、综合评分或周期性全房间调档。
|
||||
- 路径隔离表示：direct/peer 继续使用每条 PeerConnection 独立的 stock WebRTC GCC，不把 Viewer 反馈合成全房目标。Browser Host 的唯一 SFU publication 不设置 `screenShareSimulcastLayers`，由锁定的 LiveKit client 默认行为拥有 VP8 screen-share representations、source replacement 与 republish；LiveKit Dynacast 和启用 send-side BWE 的 server stream allocator 负责需求聚合与每个 subscriber 的实际下行层。Screener 不实现质量评分、带宽估计、周期调档或应用媒体层 selector。所有当前 Viewer 都可能承担 peer children，因此 subscriber 保持 `adaptiveStream: false`，不得让本地元素尺寸或可见性控制 relay ingress。表示选择不改变拓扑、端点容量或路由失效权威。
|
||||
- 路由失效输入：应用不推断 parent-wide 质量，也不计算综合分数。当前 authenticated child 只在 PeerConnection `failed/closed`，或非权威暂停时现有 periodic stats 在统一 deadline 内没有新 decoded frame 时，使自己的 exact edge 失效并唤醒同一个 reconcile。decoded progress 只保存 route-keyed `lastDecodedAt`，route/connection变化或新帧重置、暂停抑制，同一 edge只报告一次，不增加轮询器。Active SFU route 暂时没有 video track、stats report 为空或 stats 读取失败时，现有采样 cadence 仍把该次作为无 decoded progress 推进 deadline；它不得停表、伪造连接指标或立即判定 edge 失效，只有 route 停用或 teardown 才停止该 cadence。该 operation 把失败的 exact direct parent 在当前 fact version 直接视为已尝试，先选择其他 direct parent，再进入有界 SFU suffix；新的外部事实才允许原 direct tuple 重新进入。Track availability、低码率、低 FPS、模糊、sender/viewer stats 与 SFU layer只用于stock WebRTC/LiveKit恢复和诊断。relay自身ingress失效时，它作为child换父并保留subtree；disconnect或effective downstream capacity降低只让direct/overflow children逐个走同一事务。
|
||||
- 断线恢复：临时网络变化先交给 WebRTC/LiveKit 的 ICE、consent、重连和流状态。Direct edge 可在 exact current parent/child sessions 内采用 framework rebuild 的新 connection ID而不改变topology。应用只在上述exact edge失效后运行一次有界reconcile；任一步成功即停止。候选耗尽时hard-invalid、confirmed-departed或overflow edge被退休并进入明确waiting，healthy bootstrap edge保留。所有异步结果受authenticated session、share generation、base/pending route revision与candidate connection identity约束，未受影响分支保持sticky。权威暂停abort未提交candidate、保留active graph、抑制stall，并在resume后重新reconcile。
|
||||
- 同一 operation 的一个总 deadline 只按实际存在的 `direct | sfu` 语义阶段等分，不配置每个 candidate 的 magic timeout。阶段内的硬失败继续推进 cursor；silent ICE 到达 direct 阶段边界时跳过剩余 direct tuple，必须把派生份额留给 SFU。20 名 Viewer 同时加入时，每名 child 的 operation deadline 不会被人数除分，但单一 room-serial operation 可能形成线性排队尾延迟；在接受另一种并发模型前，只测量 queue wait、candidate start、first decoded frame 与 final route 的 p50/p95/max，不用 NAT 猜测、TCP probe 或 magic timeout 绕过排队。
|
||||
@@ -45,12 +45,12 @@ SFU 的 publication、subscription、server-resource accounting、临时 overlap
|
||||
- 独立房间入口：合法 `/r/{code}#v={grant}` 邀请直接进入对应房间；没有 grant 的 `/join` 或 `/r/{code}` 先显示明确标为“站点访问”的整站验证，再提交 code-only 尝试。`open` 且可准入时直接进入；未分配、已回收或已过期的 code 明确显示“房间不存在或已过期”且不提供密码输入；现存房间的其他预期拒绝使用 `ROOM_ACCESS_DENIED`，不披露具体 policy。有效 site-access cookie 只跳过第一步，两类密码不得复用或互相替代。Host 可设置、轮换或移除房间观看密码；密码接受 1 至 64 个可见 ASCII 字符，异步 scrypt 全局最多执行 2 个、等待 16 个派生。Host 可把房间密码保存在同源本地创建偏好中；它不得进入 URL、cookie、日志或服务端持久存储。
|
||||
|
||||
- 统一自动路由：旗舰部署开启 `PEER_ASSISTED_MEDIA` 后，每个房间使用一个服务端 route controller，独占 committed graph、一个 pending child operation 和 event-driven reconcile。运行时只增加一个串行pump、一个operation deadline timer与typed resource lifecycle task，不维护另一份graph。operation持有candidate list/cursor、一个current candidate/资源预留、fact version和一个总deadline。committed graph必须无环、source-reachable，且每名Viewer同时至多一个active upstream。加入、容量变化、节点离开、exact edge `failed/closed`，以及非暂停decoded-frame stall都只唤醒同一个reconcile；其余健康分支保持sticky。候选按当前身份、source reachability、无环、effective capacity与sender transition硬过滤，再按结果深度、剩余sender slots、稳定加入顺序和peer identity排序；因此 Host root退出后新释放的Host槽位比已有root的更深槽位优先，但健康旧边不因周期性平衡而迁移。server admission在执行当前cursor时独立reserve。candidate child首帧ready触发`settleReady(commitAdmissionSync)`，成功prepare `P`提交active `P`；失败释放资源并用同一monotonic allocator广播`R>P` rollback，cursor推进但总deadline不重置。bounded-gap先preflight server resource，再retire producer、按新graph重算未尝试suffix，并在切断healthy edge时追加一个普通restore candidate。relay自身ingress失效时作为child换父并保留subtree；relay disconnect/cap降低让overflow children逐个走同一事务，childless后删除。房间admission和无环约束最大深度，不设独立hard cap。普通peer始终STUN-only，SFU使用独立admission。
|
||||
- 观看端连接反馈：Viewer 舞台从鉴权到首帧使用一个由现有权威状态推导的稳定阶段反馈，至少区分等待分享、分配线路、建立 P2P、连接备用线路、等待首帧、浏览器阻止自动播放、恢复连接和有界失败。阶段不是进度百分比，也不产生 route authority；只有当前 video element 实际提交首帧后才显示正在播放。播放按钮只在 `play()` 因自动播放策略被拒绝时稳定出现，连接中不得忽隐忽现；明确失败后才提供一次受现有 controller 约束的重试入口。make-before-break 或仍有可见旧帧时保留画面并叠加恢复状态，不为显示反馈主动清空健康 media element。Host 离线本身不使仍健康的媒体失效；exact upstream 硬失败或 route terminal failure 必须使 current frame proof 失效，已证明画面至多降为 retained background，不能遮蔽离线或有界失败。已接受的 `screener-v12` 服务端补充消息是 strict、current-revision-fenced `route-status` union，且只允许 `waiting/sfu-admission` 与 `failed/route-exhausted` 两种配对；它不携带 raw text、身份、candidate 或 retry hint，prepare/active、access、Host、signal、media 与 autoplay 继续由各自权威输入派生。
|
||||
- 质量控制扩展：推荐档位仍是默认入口且只保留 1080p60/1080p30/720p30 三档，当前页面初始 desired state 选择 1080p30 且不写 `localStorage`；“分享高级设置”允许 480p（854x480）/720p/1080p/1440p 分辨率 ceiling、15 至 60 的整数 FPS ceiling、2 至 12 Mbps 视频码率 ceiling、清晰/平衡/流畅三种偏好，以及离散的 `64 kbps 省流 | 128 kbps 音乐默认 | 256 kbps 极高音质` 屏幕音频 sender ceiling；480p 不形成推荐档或固定 FPS/码率组合，初始值采用 `balanced` 与 128 kbps。视频档位和音频档位可在直播中显式应用。所有当前及以后建立的 Web Host、Browser relay 与 SFU sender 都读取最新 desired 档位，并各自在 fresh sender parameters 上应用和读回 endpoint-local applied 值；实际码率仍由浏览器拥塞控制和 stats 决定。Peer 只在唯一明确的 active Opus audio 上做结构化、幂等的 Viewer answer `stereo=1;maxaveragebitrate=256000` 变换，使同一接收上限覆盖三档；非目标形态或 parser/writer 异常必须原样保留浏览器 answer，不得阻断可继续的 mono/video edge。不得用宽泛字符串/正则改 SDP，也不得改写浏览器协商的 FEC。SFU 将三档映射到 pinned SDK 的 music stereo preset 或显式 256 kbps ceiling，统一保留 `forceStereo = true`、`dtx = false` 与 pinned room default `red = true`。不得暴露任意码率、声道、采样率、audio codec、stereo、DTX、RED 或 FEC 控件；256 kbps 只是显式较高上限，不是无损、高保真或最低码率保证。实际质量持续低于所选上限时必须显示可解释的限制原因；在真实拥塞、编码器和接收端证据完成前，不把任一组合称为“完美平衡”。未来 microphone voice 若进入范围，必须是独立 track/处理链，不能把 AEC/降噪/DTX 套到电影或游戏音频。
|
||||
- 观看端连接反馈：Viewer 舞台从鉴权到首帧使用一个由现有权威状态推导的稳定阶段反馈，至少区分等待分享、分配线路、建立 P2P、连接 SFU、等待首帧、浏览器阻止自动播放、恢复连接和有界失败。阶段不是进度百分比，也不产生 route authority；只有当前 video element 实际提交首帧后才显示正在播放。Autoplay rejection 通过原生媒体 controls 恢复；连接中不显示虚假的播放动作。手动“重新连接媒体”只重建当前 exact route：P2P 使用同一 parent，SFU 使用同一 publication/subscription，不触发主动换父、质量选优或 P2P/SFU 重选；真实恢复耗尽后才进入既有 route-failure operation。make-before-break 或仍有可见旧帧时保留画面并叠加恢复状态，不为显示反馈主动清空健康 media element。页面从冻结/隐藏恢复时先重置 decoded-stall wall-clock baseline 并重新证明当前代画面，冻结时间本身不得成为立即换路依据。Host 离线本身不使仍健康的媒体失效；exact upstream 硬失败或 route terminal failure 必须使 current frame proof 失效，已证明画面至多降为 retained background，不能遮蔽离线或有界失败。已接受的 `screener-v12` 服务端补充消息是 strict、current-revision-fenced `route-status` union，且只允许 `waiting/sfu-admission` 与 `failed/route-exhausted` 两种配对；它不携带 raw text、身份、candidate 或 retry hint，prepare/active、access、Host、signal、media 与 autoplay 继续由各自权威输入派生。
|
||||
- 质量控制扩展：推荐档位仍是默认入口且只保留 1080p60/1080p30/720p30 三档，当前页面初始 desired state 选择 1080p30 且不写 `localStorage`；“分享高级设置”允许 480p（854x480）/720p/1080p/1440p 分辨率 ceiling、15 至 60 的整数 FPS ceiling、2 至 12 Mbps 视频码率 ceiling、清晰/平衡/流畅三种偏好，以及离散的 `64 kbps 省流 | 128 kbps 音乐默认 | 256 kbps 极高音质` 屏幕音频 sender ceiling；480p 不形成推荐档或固定 FPS/码率组合，初始值采用 `balanced` 与 128 kbps。视频档位和音频档位可在直播中显式应用。所有当前及以后建立的 Web Host、Browser relay 与 SFU sender 都读取最新 desired 档位，并各自在 fresh sender parameters 上应用和读回 endpoint-local applied 值；实际码率仍由浏览器拥塞控制和 stats 决定。Peer 只在唯一明确的 active Opus audio 上做结构化、幂等的 Viewer answer `stereo=1;maxaveragebitrate=256000` 变换，使同一接收上限覆盖三档；非目标形态或 parser/writer 异常必须原样保留浏览器 answer，不得阻断可继续的 mono/video edge。不得用宽泛字符串/正则改 SDP，也不得改写浏览器协商的 FEC。SFU 将三档映射到 pinned SDK 的 music stereo preset 或显式 256 kbps ceiling，统一设置 `forceStereo = true`、`dtx = false` 与 `red = false`。不得暴露任意码率、声道、采样率、audio codec、stereo、DTX、RED 或 FEC 控件；256 kbps 只是显式较高上限，不是无损、高保真或最低码率保证。实际质量持续低于所选上限时必须显示可解释的限制原因；在真实拥塞、编码器和接收端证据完成前，不把任一组合称为“完美平衡”。未来 microphone voice 若进入范围，必须是独立 track/处理链，不能把 AEC/降噪/DTX 套到电影或游戏音频。
|
||||
- SFU 质量设置生命周期：直播中修改档位时，浏览器 sender 的 applied readback、LiveKit `LocalVideoTrack` 保存的 degradation preference、track restart options 与 `LocalTrackPublication` 的 republish options 必须作为同一次串行操作更新；任一步失败恢复旧 sender 与旧 SDK 保存态，旧 publication generation 不得回写当前状态。LiveKit full reconnect/republish 不得恢复初次发布的旧偏好。该修复不改变 codec、码率上限、层数或路由，也不能单独解释即时低帧。
|
||||
- 本地预览性能候选：在相同动态场景中 A/B 测量 Host 动态预览开/关的 CPU、GPU、encode time 和实际 FPS。只有稳定收益且捕获、编码和远端画面不受影响时，才允许直播开始后默认显示静止缩略图或隐藏动态预览并提供按需展开；未测前不宣称收益。
|
||||
- Peer-assisted 媒体路径：所有非服务器 endpoint 都受同一个服务端权威稳态 outbound media-copy capacity 约束，默认 `2` 且可静态配置为 `1`、`2` 或 `3`；peer child 与 Host publication 都按实际发送副本计数。每名 Viewer 任一时刻只有一个 active upstream，活动媒体图保持无环；只在离散事件中做有界分配或换父。浏览器把收到的标准 remote track 加入独立下游 `RTCPeerConnection` 时会逐跳解码并分别重编码，必须测量而不能称为 shared encode。服务端只在有界房间内存中保存严格校验的质量设置，不写入持久存储。
|
||||
- 原生 shared encode 分享端：后续独立的 Electron/原生分享端若获新决策，必须让每个 active representation 只产生一份兼容编码结果，并在获准的 direct、peer 与 SFU transport 间复用。Native 如何提供已接受的 SFU LOW/HIGH 表示仍须通过明确硬件路径和真实游戏、编码、上传门槛。每条连接仍保留独立 WebRTC packetization、SSRC、pacing、加密和反馈，且各自消耗上传带宽。浏览器不保证跨 `RTCPeerConnection` 复用编码器，因此当前 Web spike 不能宣称满足该保证。
|
||||
- 原生 shared encode 分享端：后续独立的 Electron/原生分享端若获新决策，必须让每个 active representation 只产生一份兼容编码结果，并在获准的 direct、peer 与 SFU transport 间复用。Native 如何满足届时锁定的 SFU representation contract 仍须通过明确硬件路径和真实游戏、编码、上传门槛。每条连接仍保留独立 WebRTC packetization、SSRC、pacing、加密和反馈，且各自消耗上传带宽。浏览器不保证跨 `RTCPeerConnection` 复用编码器，因此当前 Web spike 不能宣称满足该保证。
|
||||
- 原生 shared encode relay：具备原生 relay 能力的非服务器端点同样受统一服务端权威下游容量约束，并应复用兼容编码输出而不再次编码。当前普通浏览器 relay 使用逐边重编码的标准 WebRTC 路径，不伪称 shared encode。
|
||||
- SVC 重开条件：当前标准 Web/LiveKit SVC 已静态拒绝为 `no-go-web-svc-cross-path-hardware-contract`。sender `scalabilityMode` 没有 direct/peer receiver 对应的按层选择或跨 PeerConnection shared-encode 契约；requested/applied mode、Media Capabilities `powerEfficient` 和 RTCStats encoder 字段也不能证明没有静默软件 fallback。Pinned client 2.22.0 还会把 screen-share SVC 覆盖为三层 temporal-only `L1T3`，违反当前 Browser 无 SVC、单表示合同。只有未来 native sender 能明确选择硬件编码器、最多发布两层、将同一编码输出复用于 direct/peer/SFU、保持每条路径独立选择并通过真实游戏和亚秒延迟门槛时，才可用新的 capability decision 重开 SVC；不能把浏览器 API 支持或 UA 推断当成采用理由。
|
||||
- 分包/分层多树实验：把编码包、对象或可独立解码层拆到多个 peer 路径以分散上传，只能在独立 spike 中验证。必须保持亚秒级，处理多父重组、排序、丢包恢复、关键帧、同步和节点退出；未通过门槛前不进入产品路由控制器。
|
||||
@@ -58,7 +58,7 @@ SFU 的 publication、subscription、server-resource accounting、临时 overlap
|
||||
- 分享端桌面封装：用 Electron 固定 Chromium 版本、改善选源/热键/系统音频体验，同时保留 Web 观看端。
|
||||
- Windows 应用音频：Windows 11 Native sender 使用本地 opaque window target 绑定 HWND、PID 与进程创建时间，只捕获所选游戏及其子进程的音频，避免把语音聊天或通知声一并分享；“全部系统声音”只能由分享者显式选择。普通 Windows 10 不具备当前 process-loopback API 门槛，必须明确标为未解决/不支持而不是伪兼容。浏览器无法证明或隔离任意进程音频时必须如实提示，原生捕获失败、无声或进程退出不得静默扩大为整机捕获。
|
||||
- 受限网络边界：当前媒体只使用 ordinary direct/STUN UDP 与 LiveKit SFU/UDP；全 UDP 封锁、只允许标准 HTTPS 出站等网络在有界窗口后明确失败。不得用 TCP 探测、客户端声称的 NAT 类型或另一套应用 route candidate 猜测可达性。若真实矩阵证明该失败边界需要扩展，先比较 LiveKit 内建 ICE/TCP 或其内部安全 relay transport，并另行接受安全、端口、延迟和部署合同。
|
||||
- Web 编码：视频 track 不设置任何 `contentHint`，让 display capture 保留浏览器的 screen 分类；音频 track 继续使用 `contentHint = "music"`。Browser direct、browser relay 与 SFU publication 的唯一视频媒体 codec 是 VP8，不提供 Automatic/H.264/其他 codec 选择、fallback 或运行时切换。实际 codec/profile、encoder implementation 与 stats 继续作为只读诊断事实，不产生自动切换或路由 authority；依据由 [realtime quality research](./research/realtime-quality-adaptation.md#browser-codec-and-content-hint-evidence) 持有。
|
||||
- Web 编码：视频 track 使用标准 `contentHint = "motion"` 表达游戏/动态画面意图，音频 track 使用 `contentHint = "music"`；两者都不承诺实际 codec、分辨率、帧率、码率或硬件实现。Browser direct、browser relay 与 SFU publication 的唯一视频媒体 codec 是 VP8，不提供 Automatic/H.264/其他 codec 选择、fallback 或运行时切换。实际 codec/profile、encoder implementation 与 stats 继续作为只读诊断事实，不产生自动切换或路由 authority；依据由 [realtime quality research](./research/realtime-quality-adaptation.md) 持有。
|
||||
|
||||
### P2 - 可选功能
|
||||
|
||||
@@ -126,11 +126,11 @@ SFU 的 publication、subscription、server-resource accounting、临时 overlap
|
||||
|
||||
- Given 观看者只有受支持的手机浏览器，When 打开有效邀请链接并执行浏览器要求的播放手势，Then 无需安装应用即可进入房间、看到首帧并听到可用音频。
|
||||
- Given 用户在普通手机浏览器尝试成为 Host，When `navigator.mediaDevices.getDisplayMedia` 缺失，Then 页面在创建发布路径前明确显示该浏览器不支持移动端屏幕分享，并保留加入房间入口。Given API 存在但真实调用失败，When 失败原因是拒绝/取消或其他 capture error，Then 分别显示对应状态而不是一律重标为不支持；判断使用运行时能力/结果而不是 UA、平台或设备型号，也不能把移动 Viewer 支持宣传成移动 Web Host 支持。
|
||||
- Given 观看者调节音量或静音，When 媒体切流、换父或恢复连接，Then 当前页面的本地播放音量保持不变，其他观看者和发送端不受影响；刷新页面后恢复默认音量，不产生信令、持久化或媒体重协商。
|
||||
- Given 观看者操作浏览器原生媒体 controls，When 播放、暂停、调节音量、静音或进入全屏，Then 只改变当前 video element，其他观看者、发送端、房间与路由不受影响，不产生信令、应用持久化或媒体重协商。
|
||||
- Given Viewer 已通过浏览器要求的用户手势开始可听播放，When 页面切到后台且浏览器仍保留该页面，Then 同一个媒体元素、信令和 WebRTC 连接继续承载音频，返回前台后沿当前直播恢复画面而无需重新加入；后台视频合成或逐帧回调增长不作为连续播放承诺。Given iOS 锁屏、系统回收页面或后台期间产生新的媒体连接，Then 以当前真机行为为准，恢复时继续使用已有重连和显式播放入口。
|
||||
- Given 用户查看本机身份，When 尚未进入编辑状态，Then 界面显示“昵称”、当前昵称和紧贴其后的铅笔按钮，不显示默认可编辑输入框。Given 用户修改昵称后以 Host 或 Viewer 加入房间，Then 所有在线参与者看到同一份服务端权威 roster、对应房间标题；唯一名称只显示名称，重名项追加当前房间 `peerId` 短后缀。刷新后本机名称仍在，清除站点数据后恢复未命名默认值，服务器不存在显示名或成员的持久记录，展示身份不参与授权或路由。
|
||||
- Given 手机从 Wi-Fi 切换到蜂窝网络或页面从后台恢复，When 原候选对失效，Then 客户端执行 ICE restart 或重建连接，并显示恢复进度而不是永久黑屏。
|
||||
- Given Viewer 已鉴权但尚未提交首帧，When route 从 peer prepare 进入 SFU fallback 或等待媒体，Then 固定舞台依次显示当前真实阶段且不跳动、不显示虚假百分比；只有当前代 video frame 被 compositor 接收后才显示正在播放。Given `video.play()` 仅因 autoplay policy 被拒绝，Then 稳定显示一个播放动作；连接尚未完成时不显示忽隐忽现的播放按钮。Given 旧媒体仍可见且正在 MBB 恢复，Then 保留旧画面并叠加恢复状态。
|
||||
- Given Viewer 已鉴权但尚未提交首帧，When route 从 peer prepare 进入 SFU 或等待媒体，Then 固定舞台依次显示当前真实阶段且不跳动、不显示虚假百分比；只有当前代 video frame 被 compositor 接收后才显示正在播放。Given `video.play()` 仅因 autoplay policy 被拒绝，Then 保留原生媒体 controls 并显示非阻塞播放暂停状态；连接尚未完成时不显示自制播放按钮。Given 旧媒体仍可见且正在 MBB 恢复，Then 保留旧画面并叠加恢复状态。
|
||||
- Given Viewer 尝试把直播输出到电视，When 浏览器/系统提供已验证的 Remote Playback、AirPlay 或屏幕镜像能力，Then 该动作只改变本机输出设备，现有 upstream PeerConnection、SFU/peer route 和其他参与者保持不变。Given live `srcObject` 未经目标设备证明或 API 缺失，Then 产品不显示伪可用 Cast/AirPlay 按钮，也不创建自有 receiver 或转码服务器。
|
||||
|
||||
### 一对多与带宽保护
|
||||
@@ -151,16 +151,16 @@ SFU 的 publication、subscription、server-resource accounting、临时 overlap
|
||||
### 质量和恢复
|
||||
|
||||
- Given 用户选择任一推荐档位或高级质量参数，When 接收端实际分辨率、帧率或码率持续显著低于该目标，Then 诊断数据能区分采集约束、发送编码、带宽/拥塞、当前 peer/SFU 路径和接收端限制；产品不得只改变档位标签却继续隐藏实际降级。
|
||||
- Given Browser Host 建立 SFU publication，When sender 参数和 RTCStats 可读，Then 最多存在固定的半分辨率与原始分辨率两个 VP8 outbound video encoding，表示数不随 Viewer 数增长；LiveKit Dynacast 按聚合层需求停用其上的层，启用 send-side BWE 的 server stream allocator 为每个 SFU subscriber 选择实际下行层，弱下行不能改变健康路径、拓扑或 direct/peer 各自的 stock WebRTC GCC。
|
||||
- Given Browser Host 建立 SFU publication，When publish options、sender 参数和 RTCStats 可读，Then Screener 不设置自定义 simulcast layer list，实际 VP8 encodings 来自 pinned LiveKit 默认行为且不随 Viewer 数线性创建；LiveKit Dynacast 与启用 send-side BWE 的 server stream allocator 负责聚合需求和每个 SFU subscriber 的实际下行层，弱下行不能改变健康路径、拓扑或 direct/peer 各自的 stock WebRTC GCC。
|
||||
- Given 当前 Viewer 的 exact edge 进入 `failed/closed`，或在非权威暂停时超过统一 deadline 没有新 decoded frame，When 服务端确认 authenticated child、share generation、active revision 与 connection identity 仍匹配，Then 只把该 child 送入同一个 reserve/prepare/first-frame/commit-or-abort 事务；低码率、低 FPS、模糊和 SFU layer 只进入诊断。
|
||||
- Given 当前 edge 仍在解码但分辨率、FPS、码率、freeze ratio、RTT、jitter、loss 或 limitation evidence 持续影响观感，When 评估是否自动换父，Then 当前产品只诊断而不迁移：现有路径证据不能证明未连接 parent 的反事实质量，WebRTC/LiveKit 继续拥有拥塞适应，应用不建立 alternative-parent probe、hysteresis/cooldown、加权分数或周期性重排。
|
||||
- Given 同一 Viewer 同时是某人的 child 和其他人的 parent，When 自身 ingress 失效，Then 它作为 child 换父、在新 ingress 首帧提交前保留可用旧 edge，并保持 subtree。Given endpoint disconnect 或 `effectiveDownstreamCapacity` 降低，Then controller 只保留稳定顺序内可容纳的 children，并将 deterministic overflow children 逐个送入同一事务；childless 后删除 disconnected relay。
|
||||
- Given 当前分享权威 paused，When 新 Viewer 加入、edge 失效或已有 child operation 尚未提交，Then abort 整个 pending route operation 及其 current candidate/预留、保留 active graph、让新 Viewer waiting，并且不因没有 decoded frame 触发恢复。普通 Pause/Resume 只通过当前 Host session 与 `shareGeneration` 约束的一次 paused-state 更新同步本地 capture tracks、服务端快照、`host-status` 和 route pause；Resume 重新启用同一批 tracks并从当前 graph 唤醒 fresh reconciliation，不建立第二次授权、source acknowledgement、媒体 proof 或回滚状态。较早 share generation、已替换 Host session、stop 或新分享的迟到更新不得改变当前状态。
|
||||
- Given 当前 Browser SFU publication，When 进行质量验收，Then 受限 SFU subscriber 必须能由 LiveKit 原生选择半分辨率表示，非受限 subscriber 仍可取得原始分辨率表示；Dynacast 在只有较低需求或无订阅时停用更高表示或全部表示，但任何最高需求允许两路同时编码。不得退回全房间最低目标、启用 AdaptiveStream 控制潜在 relay ingress，或增加应用媒体层 controller。
|
||||
- Given 当前 Browser SFU publication，When 进行质量验收，Then 受限与非受限 subscriber 的实际 representation 由 pinned LiveKit 原生选择并以 sender/server/receiver stats 证明；不得把一次 layer 数量或参数写成 Screener 产品常量，不得退回全房间最低目标、启用 AdaptiveStream 控制潜在 relay ingress，或增加应用媒体层 controller。
|
||||
- Given 同一时刻采集、host outbound 与 viewer inbound 证据，When 进行根因分类，Then 关联派生后的实际协商视频 codec/profile/parameters、适用时的 `scalabilityMode` 与真实解码表现，依次区分 Capture 低、Capture 高但 outbound 因 CPU 低、Capture 高但 outbound 因 bandwidth 低、outbound 正常但 inbound 低、以及 inbound 正常但肉眼模糊五类；不用 UA 或设备型号代替运行时证据，也不上传 raw SDP。
|
||||
- Given Viewer 上报当前路径质量证据，When 报告未鉴权、过频、重复、过期或没有匹配的 sender/viewer 统计，Then 报告被丢弃；有效报告也只能用于诊断，不能改变 Host 表示或路径状态。
|
||||
- Given 受控 RTT/丢包测试条件，When 连续分享高动态游戏画面 30 分钟，Then 延迟、帧率与丢帧达到上述目标，且没有不可恢复的冻结。
|
||||
- Given Windows Chrome/Edge Browser Host 开始或继续分享，When 浏览器返回 display video track，Then 应用不赋任何视频 `contentHint`，并按浏览器 stats 如实显示实际 RTP codec 与实现字段。Given 已接受的 `screener-v12` 建立 direct 或 browser-relay sender，Then offer 的唯一视频媒体 codec 是 VP8；RTX/RED/FEC 只可作为修复格式，不能引入另一媒体 codec。若浏览器缺少 VP8 capability、`setCodecPreferences()` 或应用失败，Then 在创建 offer 前明确失败而不是静默协商其他 codec。Given 建立 SFU publication，Then LiveKit 固定使用 VP8 且不启用 backup codec。
|
||||
- Given Windows Chrome/Edge Browser Host 开始或继续分享，When 浏览器返回 display video track，Then 应用设置 `contentHint = "motion"`，并按浏览器 stats 如实显示实际 RTP codec 与实现字段。Given 已接受的 `screener-v12` 建立 direct 或 browser-relay sender，Then offer 的唯一视频媒体 codec 是 VP8；RTX/RED/FEC 只可作为修复格式，不能引入另一媒体 codec。若浏览器缺少 VP8 capability、`setCodecPreferences()` 或应用失败，Then 在创建 offer 前明确失败而不是静默协商其他 codec。Given 建立 SFU publication，Then LiveKit 固定使用 VP8、不启用 backup codec，且 Screener 不设置自定义 simulcast layer list。
|
||||
- Given 分享过程中切换 Wi-Fi/有线网络，When ICE 连接失败，Then 客户端触发 ICE restart，并在成功或超时后给出明确状态。
|
||||
- Given 当前路径仍可播放且控制器准备迁移 logical parent 或替换 transport，When 候选路径拥有模型授权的 overlap 资源，Then 旧路径保持权威，直到 exact candidate child 解码出第一帧新画面后才原子提交；ICE `connected` 单独不能触发切换。
|
||||
- Given 候选没有准入资源，或 provisional 路径失败、超时、代次过期、参与者变化、房间停止、权限撤销或 session 被替换，When 本次机会结束，Then 原路径和未受影响 branches 继续工作，临时资源被销毁，重试遵守模型定义的有界策略。
|
||||
@@ -168,7 +168,7 @@ SFU 的 publication、subscription、server-resource accounting、临时 overlap
|
||||
- Given 分享者选择标签页或窗口，When 浏览器支持对应音频捕获，Then picker 提供该来源音频而不扩大为系统混音。Given 分享者选择整个屏幕，When 浏览器支持系统音频，Then picker 提供由用户决定的系统音频开关。应用同时请求 `windowAudio: "window"` 和 `systemAudio: "include"`，不根据 track 存在性伪造来源标签。
|
||||
- Given 浏览器返回分享音轨，When 开始发布，Then 应用设置标准 `contentHint = "music"` 作为内容意图元数据；该值可读回，但不作为 Opus、码率、声道、采样率、FEC、DTX 或立体声的质量证据。
|
||||
- Given Web Host 或浏览器 relay 建立或替换分享音轨，When 对实际音频 sender 应用发送参数，Then 从当次 `getParameters()` 派生所选 `encodings[0].maxBitrate = 64000 | 128000 | 256000`、调用 `setParameters()` 并立即读回，旧配置缺失时按 128000 处理；Given Web Viewer 为含唯一有效 audio/Opus payload 的 offer 生成 answer，Then 用结构化 SDP parser 只 upsert 该 payload 的 `stereo=1` 与 `maxaveragebitrate=256000`，并把同一个结果用于 `setLocalDescription()` 和 signaling。首次连接、ICE restart、同 parent rebuild 和 relay child 都适用；空 SDP、无/歧义 Opus、多个有效 audio、重复目标参数、畸形 fmtp 或 parse/write 异常必须按对象 identity 返回浏览器原 answer 并继续，不能发送部分修改或静默冒充 stereo/实际码率。只有实际 `setLocalDescription()` 失败才进入既有连接错误路径。`sprop-stereo` 与 `usedtx` 不新增，其他 fmtp/FEC/session/media 字段逐项保留。
|
||||
- Given 通过 SFU 发布分享音轨，When 构建 LiveKit 2.22.0 publish options，Then 64 kbps 使用 `AudioPresets.musicStereo`、128 kbps 使用 `AudioPresets.musicHighQualityStereo`、256 kbps 使用同等结构的显式 sender ceiling，三档都设置 `forceStereo = true` 与 `dtx = false`，且不覆盖 pinned room default `red = true`；验收同时核对 SDK 选项、applied ceiling、当前 RTP codec/fmtp 与 stats 实际码率，不能从 preset 名称或 `opus/48000/2` 单独推断实际声道内容。
|
||||
- Given 通过 SFU 发布分享音轨，When 构建 LiveKit 2.22.0 publish options，Then 64 kbps 使用 `AudioPresets.musicStereo`、128 kbps 使用 `AudioPresets.musicHighQualityStereo`、256 kbps 使用同等结构的显式 sender ceiling，三档都设置 `forceStereo = true`、`dtx = false` 与 `red = false`；验收同时核对 SDK 选项、applied ceiling、当前 RTP codec/fmtp 与 stats 实际码率，不能从 preset 名称或 `opus/48000/2` 单独推断实际声道内容。
|
||||
- Given 分享正在直播或暂停且 Host 选择新的 64/128/256 kbps 音频档位，When 当前 profile mutation 串行执行，Then 服务端以 last-wins 保存并广播最新 desired profile，每个当前 Host、relay 与 SFU audio sender 从 fresh `getParameters()` 派生 `setParameters()` 并立即 readback，过期 completion 不得覆盖新值；以后创建的 sender 使用最新 desired profile。单个端点应用失败时保留媒体与其旧 applied 值、显示可见告警并允许后续更新或重建重试，不得谎称全房已收敛。当前 applied readback 保持 endpoint-local；没有显式回执 wire 就不宣称跨端点原子提交。该操作不改变 Opus、route revision、media-binding generation 或 publication generation，也不创建第二个音频控制环。当前 screen-audio Host 基线是 Chrome/Edge；其他浏览器只有实际 sender readback 成功时才算支持。
|
||||
- Given Host 打开已接受的 `screener-v12` 分享设置或更新房间画质，When 查看 UI 或校验消息，Then 不存在 codec selector、codec quality-state/wire 字段或分享生命周期 codec 分支；Browser direct、browser relay 与 SFU publication 始终使用固定 VP8。实际协商 codec 与 encoder 只作为只读诊断显示。
|
||||
- Given 浏览器返回屏幕音轨，When 产品展示音频状态或未来连接详情，Then 只能把存在性、浏览器实际返回的 track settings、实际协商 codec/fmtp 和实际流量分别标为捕获、协商或观测事实；字段缺失必须显示为未知，且不得从 Opus 的 `48000/2` RTP 声明推断源采样率或实际 stereo。
|
||||
|
||||
Reference in new issue
Block a user