docs(media): define demand-driven quality fallback (#31)
docs(media): define demand-driven quality fallback
This commit is contained in:
commit
acba8973d6
10 files changed
+385
-44
No files matched your search
+2
-1
@@ -11,12 +11,13 @@
|
||||
- [Low-server-cost media routes](./research/low-server-media-routes.md): traffic accounting, route screening, token-free SFU standby prewarm, conditional native encoded-RTP relay, and automatic final fallback.
|
||||
- [Advanced peer distribution](./research/advanced-peer-distribution.md): multi-tree/SVC, encoded-object relay, native RTP forwarding, FEC/network-coding, MoQ, and measurable go/no-go gates.
|
||||
- [Native shared-encode sender](./research/native-shared-encode-sender.md): consolidated Draft #16/#18/#22/#23/#25/#28 evidence ladder, current Pion/WebCodecs verdicts, and product stop line.
|
||||
- [Realtime quality adaptation](./research/realtime-quality-adaptation.md): browser degradation policy, live profile changes, offline-encoding boundaries, and measurement gates.
|
||||
- [Realtime quality adaptation](./research/realtime-quality-adaptation.md): correlated capture/send/receive diagnosis, demand-driven dual representations, SVC boundary, and measurement gates.
|
||||
- [Browser screen-audio quality](./research/browser-screen-audio-quality.md): capture/source compatibility, standard readback limits, Opus negotiation boundaries, and the retained no-control decision.
|
||||
- [Agent context governance research](./research/agent-context-governance.md): official Codex, Claude Code, Hermes Agent, and GitHub practices adopted by this repository.
|
||||
- [ADR-0001](./adr/0001-p2p-first-media-topology.md): accepted P2P-first media topology and its consequences.
|
||||
- [ADR-0002](./adr/0002-persistent-protected-rooms.md): optional SQLite persistence, protected sequential room IDs, and room lifecycle.
|
||||
- [ADR-0004](./adr/0004-peer-assisted-media-experiment.md): proposed, experiment-only standard-WebRTC peer-assisted spike with hard fanout and failure gates.
|
||||
- [ADR-0005](./adr/0005-automatic-hybrid-media-routing.md): proposed, default-off automatic P2P/peer/SFU preparation, token-free standby prewarm, commit, failback, and rollback with a hard host-edge budget.
|
||||
- [ADR-0007](./adr/0007-demand-driven-dual-representation-quality.md): accepted demand-driven `HIGH + optional LOW` quality policy, path-level hysteresis, evidence contract, and SVC boundary.
|
||||
|
||||
Documentation and project memory are part of the product source of truth. Update them in the same change that alters the corresponding requirement or architecture.
|
||||
@@ -200,6 +200,9 @@ Negative:
|
||||
whole-room SFU history. ADR-0005 and merged PR #17 own the default-off
|
||||
automatic optional-SFU fallback; this ADR does not accept that route for
|
||||
production.
|
||||
- ADR-0007 owns demand-driven per-path quality representations. Its accepted
|
||||
product target does not add SVC, multiple representations, or adaptive
|
||||
switching to this bounded browser-relay experiment.
|
||||
- If a later ADR accepts peer-assisted media, it must state exactly which parts
|
||||
of ADR-0001 and ADR-0005 it supersedes.
|
||||
|
||||
|
||||
@@ -239,6 +239,8 @@ The first controller reacts only to discrete events:
|
||||
|
||||
It does not combine RTT, CPU, bitrate, geography, or a synthetic health score.
|
||||
Measurements may add a new explicit trigger only through a reviewed change.
|
||||
ADR-0007's `HIGH`/`FALLBACK` quality state is separate and does not become a
|
||||
topology trigger or a room-wide health score.
|
||||
|
||||
`MAX_SFU_ROOTS_PER_ROOM` bounds server egress. Exhausting that budget waits or
|
||||
fails explicitly; it never creates a third host edge or silently fans the SFU
|
||||
@@ -323,6 +325,8 @@ Negative:
|
||||
ADR-0003 remains historical rejected/superseded context.
|
||||
- Native shared encoding and two-tree striped distribution remain orthogonal
|
||||
data-plane experiments under separate decisions.
|
||||
- ADR-0007 owns per-path quality fallback and its two-representation budget;
|
||||
this route controller does not aggregate viewer quality into a shared target.
|
||||
|
||||
## References
|
||||
|
||||
|
||||
@@ -0,0 +1,170 @@
|
||||
# ADR-0007: Demand-Driven Dual-Representation Quality
|
||||
|
||||
- Status: Accepted - Implementation Pending
|
||||
- Date: 2026-08-19
|
||||
|
||||
## Context
|
||||
|
||||
Screener needs to protect a healthy viewer from another viewer's weak network or
|
||||
decoder without growing host encode work with viewer count. Draft PR #28 proved
|
||||
that one `VideoEncoder` object can be reconfigured from feedback shared by two
|
||||
Pion/WebRTC legs. Its minimum-of-two target was deliberately conservative for a
|
||||
bounded experiment. It is not an acceptable product policy: a weak leg would
|
||||
lower the shared stream for every healthy leg, and the experiment did not test
|
||||
heterogeneous estimates.
|
||||
|
||||
A single ordinary, non-scalable encoded representation has one resolution,
|
||||
frame cadence, and rate-control result. Forwarding those packets cannot create
|
||||
a second quality. Per-viewer quality therefore requires another encoded
|
||||
representation, a negotiated scalable representation, or transcoding at a
|
||||
relay/SFU. Per-viewer encoders would violate the game's performance budget.
|
||||
|
||||
## Decision
|
||||
|
||||
Use demand-driven dual representations:
|
||||
|
||||
1. A healthy room runs one shared `HIGH` representation.
|
||||
2. Every viewing path starts in `HIGH`. It moves to `FALLBACK` only after
|
||||
multiple consecutive sampling windows show that the path cannot sustain
|
||||
`HIGH`, using correlated sender and viewer evidence.
|
||||
3. The first verified fallback path starts one shared `LOW` representation.
|
||||
Every verified weak path uses that same representation. Healthy paths remain
|
||||
on `HIGH`.
|
||||
4. A fallback path returns to `HIGH` only after a longer, independently defined
|
||||
stable-recovery window. When no path needs `LOW`, stop its encoder and free
|
||||
its resources.
|
||||
5. The representation count is a hard `HIGH + optional LOW <= 2`. It never
|
||||
grows with viewer count.
|
||||
|
||||
Starting `LOW` is conditional on a qualified hardware/power-efficient encoder
|
||||
path and measured spare game-performance budget. If that path is absent or the
|
||||
second encoder causes unacceptable CPU/GPU/game load, fail closed: preserve
|
||||
`HIGH` for healthy paths and show the weak path an explicit degraded/unavailable
|
||||
state. Never protect a weak path by reducing healthy paths.
|
||||
|
||||
The two path states use explicit predicates and asymmetric entry/exit windows.
|
||||
There is no weighted score, device ranking, machine-learning controller, or
|
||||
continuous room-wide optimizer. Exact thresholds remain implementation inputs
|
||||
until the controlled quality matrix establishes them.
|
||||
|
||||
The topology and representation budgets are independent invariants. The host
|
||||
still has at most two downstream media edges. A native relay forwards selected
|
||||
encoded packets without decoding or re-encoding. Future dual-tree or striped
|
||||
delivery may reduce host upload from about two full copies toward one copy plus
|
||||
necessary redundancy, but it does not block this decision.
|
||||
|
||||
## Evidence Contract
|
||||
|
||||
Before implementing automatic switching, correlate the same time interval and
|
||||
stream generation across three stages:
|
||||
|
||||
| Stage | Required evidence |
|
||||
| --- | --- |
|
||||
| A. Capture | `MediaStreamTrack.getSettings()` width, height, and frame rate |
|
||||
| B. Host outbound | actual width/FPS/bitrate, target or available bitrate when present, interval encode time, `qualityLimitationReason`, RTT, loss/retransmission, selected direct/TURN route, and derived negotiated video codec/profile/parameters plus `scalabilityMode` when applicable |
|
||||
| C. Viewer inbound | actual width/FPS/bitrate, loss, jitter, interval decode/drop/freeze evidence, the corresponding derived codec/profile/parameters and applicable `scalabilityMode`, and actual decode behavior |
|
||||
|
||||
Interpretation is deliberately ordered:
|
||||
|
||||
| Observation | Classification |
|
||||
| --- | --- |
|
||||
| Capture is already low | capture or constraint problem |
|
||||
| Capture high, outbound low, reason `cpu` | sender encode/resource pressure |
|
||||
| Capture high, outbound low, reason `bandwidth` | congestion, uplink, GCC, or TURN/path pressure |
|
||||
| Outbound healthy, inbound degraded | transport or receiver-path problem |
|
||||
| Inbound metrics healthy, image still blurry | insufficient bitrate/quantization, negotiated codec/profile/layer, or display scaling problem |
|
||||
|
||||
WebRTC stats are sampled dictionaries and many useful counters are cumulative.
|
||||
The controller must use adjacent, non-overlapping deltas and rebase on a missing
|
||||
field, stream/stat ID change, or counter reset. Missing values remain unknown;
|
||||
they are not zeros.
|
||||
|
||||
Implementation is staged. First correlate A and B locally in one sampling tick,
|
||||
with an explicit interval, media/stat identity, and valid deltas. Only after
|
||||
that probe is trustworthy may the product add a minimal authenticated C report
|
||||
for the few receive/decode signals required by the two-state predicate. The B/C
|
||||
correlation uses normalized fields derived from negotiated parameters/stats and
|
||||
actual decode behavior; it must never upload raw SDP, raw stats, candidate
|
||||
addresses, or other identifiers. Do not build a general telemetry schema.
|
||||
|
||||
A viewer may request `LOW`, but the request is advisory. It must be carried on
|
||||
an authenticated, current room/path session and be rate-limited and deduplicated.
|
||||
The sender/controller accepts it only when viewer receive/decode behavior and
|
||||
derived negotiated codec/profile/parameters (including applicable
|
||||
`scalabilityMode`) agree with sender transport/GCC and encode evidence. A
|
||||
request alone cannot lower quality or start a representation.
|
||||
|
||||
UA, platform, and device-model detection does not participate in quality
|
||||
selection. The existing mobile/iPad heuristic remains limited to conservative
|
||||
relay-capacity admission. Browser capability queries can guide a bounded probe,
|
||||
but runtime encode/decode behavior is authoritative.
|
||||
|
||||
## SVC Boundary
|
||||
|
||||
Consider SVC only when a later accepted requirement demands one encoded output
|
||||
at all times. A path may then receive only the base layer or the base plus
|
||||
enhancement layers. Enable it only when the exact negotiated codec and
|
||||
`scalabilityMode` are read back, a power-efficient/hardware path is positively
|
||||
established on the supported sender cohort, and game-performance and latency
|
||||
gates pass. Do not silently fall back to software SVC. Persistent SVC is not the
|
||||
default design.
|
||||
|
||||
The WebRTC-SVC specification exposes `scalabilityMode` and permits the browser
|
||||
to return a different configured mode after negotiation. Media Capabilities can
|
||||
report support, expected smoothness, and power efficiency for a specific
|
||||
configuration. WebCodecs `hardwareAcceleration` is only a hint that a user
|
||||
agent may ignore. These APIs are useful evidence, not a portable guarantee of a
|
||||
particular physical hardware encoder.
|
||||
|
||||
## Consequences
|
||||
|
||||
Positive:
|
||||
|
||||
- Healthy viewers are not reduced to the worst path.
|
||||
- Normal operation pays for one encoder; a second is paid only while at least
|
||||
one verified weak path needs it.
|
||||
- Encode cost is bounded independently of viewer count.
|
||||
- The state machine and its evidence remain inspectable.
|
||||
|
||||
Negative:
|
||||
|
||||
- A weak-path interval can temporarily require two encoder instances and the
|
||||
bandwidth for both representations on the affected distribution edges.
|
||||
- Starting/stopping `LOW` and switching paths need a keyframe-safe transition.
|
||||
- Browser capability reporting cannot by itself prove hardware or sustained
|
||||
decode performance, so the real-device matrix remains mandatory.
|
||||
|
||||
## Product Stop Lines
|
||||
|
||||
- Do not productize PR #28's minimum-of-two bitrate or frame-rate aggregation.
|
||||
- Do not lower `HIGH` because one viewer reports trouble.
|
||||
- Do not create one representation or encoder per viewer.
|
||||
- Do not use UA/device identity as a quality signal.
|
||||
- Do not enable software SVC as an invisible fallback.
|
||||
- Do not start `LOW` when its measured resource cost violates the game budget.
|
||||
- Do not add a composite health score when the two-state predicates suffice.
|
||||
|
||||
## Verification Gates
|
||||
|
||||
- A healthy room emits only `HIGH`.
|
||||
- One sustained weak path starts exactly one `LOW`; healthy viewers remain on
|
||||
`HIGH`, and additional weak viewers reuse `LOW`.
|
||||
- After all weak paths sustain the longer recovery window, every path returns
|
||||
to `HIGH` and the `LOW` encoder stops.
|
||||
- Representation count never exceeds two and host media edges never exceed two.
|
||||
- An unavailable/over-budget `LOW` fails visibly for weak paths while healthy
|
||||
paths and `HIGH` remain unchanged.
|
||||
- Native relays show packet forwarding without an added decode/encode stage.
|
||||
- Spoofed, stale, duplicated, or rate-excessive viewer requests have no effect.
|
||||
- Controlled capture/CPU/bandwidth/TURN/receiver/display cases produce the
|
||||
classifications in the evidence table without a room-wide downgrade.
|
||||
|
||||
## References
|
||||
|
||||
Primary sources checked 2026-08-19:
|
||||
|
||||
- [W3C WebRTC Statistics](https://www.w3.org/TR/webrtc-stats/)
|
||||
- [W3C Media Capture and Streams](https://www.w3.org/TR/mediacapture-streams/)
|
||||
- [W3C WebRTC SVC](https://www.w3.org/TR/webrtc-svc/)
|
||||
- [W3C Media Capabilities](https://www.w3.org/TR/media-capabilities/)
|
||||
- [W3C WebCodecs](https://www.w3.org/TR/webcodecs/)
|
||||
@@ -10,7 +10,7 @@ Last updated: 2026-08-19
|
||||
- Server bandwidth cost is a primary constraint. The ordered media ladder is direct P2P, then measured peer assistance, then an enabled user-operated or central SFU fallback. Route selection and failure recovery must be automatic and viewer-transparent, while remaining deterministic, budget-driven, and small enough to reason about. This has always been the product intent; earlier text prohibiting silent migration was an incorrect interpretation and is superseded.
|
||||
- Host and relay-capable endpoints each have a target budget of at most two downstream media edges. Every P2P edge independently uses direct ICE or authenticated TURN fallback. With two edges, the long-term native/encoded-object path should reuse one compatible encoded output; the current browser experiment is stricter at one viewer child and re-encodes.
|
||||
- Packet or layer striping across two peer trees is a recorded candidate for reducing endpoint upload toward one full-stream bitrate. It requires a separate bounded experiment for multi-parent assembly, loss recovery, synchronization, churn, and sub-second latency; it is not part of the current full-stream two-chain implementation.
|
||||
- Shared encoding reduces compute, not each viewer's last-hop copy.
|
||||
- Shared encoding reduces compute, not each viewer's last-hop copy. Quality uses shared `HIGH` plus at most one on-demand `LOW`; without a qualified efficient second encoder, weak paths fail visibly while `HIGH` remains protected. SVC has no software fallback.
|
||||
- Durable decisions, current snapshots, research, code, `AGENTS.md`, and `.codex/` belong in Git. Memory and status are rewritten in place rather than kept as transcripts.
|
||||
- Research current official sources and established implementations before material work. Apply Occam's razor and reject speculative protocols, services, scoring, and scale.
|
||||
|
||||
@@ -24,7 +24,7 @@ Last updated: 2026-08-19
|
||||
- Keep user-operated or central single-node SFU capacity optional. When configured, `main`'s #17 route controller may select it only as the final fallback; its fanout egress belongs to its operator and does not justify Redis or multi-node infrastructure. Closed PR #12's explicit whole-room SFU mode is superseded, not a current option.
|
||||
- Keep automatic routing default-off behind ADR-0005: discrete failures drive peer recovery and optional SFU fallback, host media uses break-before-make under two active edges, and a binary relay capability keeps mobile/iPad clients as leaves. Merged PR #20's token-free prewarm passes the local sub-second gate; public transport and load gates remain.
|
||||
- Prefer direct UDP, then TURN/UDP, with TURN/TCP as the required non-UDP fallback. Optional TURN/TLS uses TCP 5349 by default; TCP 443 needs a dedicated address or validated L4/SNI routing.
|
||||
- Treat video settings as ceilings. Production now defaults clarity-first with bounded manual controls and sender readback, but reported degradation remains unclassified. Keep browser audio request/presence-only: capture channels/rate are not portable controls, `maxBitrate` cannot raise quality, and Opus stereo/DTX/FEC lack portable setters. Run the 1/2/3-viewer plus TURN matrix before changing ceilings or adding an automatic controller.
|
||||
- Treat video settings as ceilings. Production defaults clarity-first with bounded manual controls, but reported degradation is unclassified. Before ADR-0007 implementation, correlate same-tick capture settings and outbound deltas/identity, then add only the minimal authenticated viewer report needed for per-path `HIGH`/`FALLBACK`. Do not use UA for quality or build a composite score. Keep browser audio request/presence-only.
|
||||
- Do not add custom scene detection or dynamic-FPS control until WebRTC statistics and host resource measurements prove a material gap. Keep browser codec order until target hardware proves a more efficient common codec.
|
||||
- Keep room policy deployment-driven. Public and password-only deployments use random temporary rooms. A site password plus SQLite path enables sequential persistent rooms; stopping sharing leaves the room and viewer link available.
|
||||
|
||||
@@ -39,11 +39,11 @@ Last updated: 2026-08-19
|
||||
- Production contains the default-off ADR-0005 controller but has no peer/SFU configuration, so it remains P2P/TURN. Repository configuration can restrict enabled peer/SFU routing to exact `PEER_ASSISTED_ROOM_IDS`; unlisted rooms stay legacy P2P, while empty or missing means all rooms.
|
||||
- When enabled later, peer recovery precedes an allowlisted SFU root; session-bound revisions prepare/commit/abort, and an active SFU gets one token refresh before share-local failback. Browser relays re-encode; native shared encoding remains outside product code.
|
||||
- Corrected Chrome 151/LiveKit 1.13.5 same-leaf/two-root cold/standby A/B reduced failure-to-active from 1.481 seconds to 200 ms and failure-to-render from 2.257 seconds to 319.7 ms. The leaf decoded/rendered 31 new frames and 25 ms sampling kept host edge peak two. It is localhost, headless, synthetic 720p30 video only; the roughly 86% result combines SDK download/parse and network prewarm and must not be extrapolated to public networks.
|
||||
- Draft native ladder #16/#18/#22/#23/#25/#28 reaches one bounded Chrome 720p30 loop: one WebCodecs object feeds two Pion/browser legs, applies their lower stock-GCC target, and recovers one isolated loss via primary-SSRC replay. This is not proof of one physical or hardware encode.
|
||||
- Draft native ladder #16/#18/#22/#23/#25/#28 reaches one bounded Chrome 720p30 loop: one WebCodecs object feeds two Pion/browser legs, applies a conservative minimum stock-GCC target, and recovers one isolated loss. The minimum policy is experiment-only and forbidden in product; it is not proof of one physical/hardware encode.
|
||||
- Stock Pion GCC plus negotiated RTX is a no-go. The no-RTX candidate can distort loss statistics and stays outside product code until audio, heterogeneous estimates, broader loss, TURN/reconnect, and browser-diversity gates pass; do not add custom congestion control to bypass them.
|
||||
- Chrome 151 short synthetic `1/3/5/8` runs kept host edges at two, relay edges at one, and all viewers decoding; a three-viewer relay close recovered in 5.32 seconds. This proves topology/control only, not quality, load, endurance, or silent partitions.
|
||||
- The current quality stack uses one strict, non-persistent `QualitySettings` value: 720p/1080p/1440p, integer 15-60 fps, 2-12 Mbps, and clarity/balanced/fluid preference. Host-only updates are last-wins for current/future relays and optional SFU; ordinary P2P wire stays unchanged. Initial, replaced, and updated senders share serialized configure/readback; rejection, rewriting, or three identical native limitation samples produce one visible warning without automatic action. Successful replacement rollback retains the original failure warning.
|
||||
- Chrome 151 with one host, three viewers, and synthetic 720p30 passed balanced and clarity propagation to every participant. Every baseline active video sender showed matching preference readback, peer fingerprints stayed stable, and every viewer decoded and rendered new frames after each change; host fanout was two and relay fanout one. This is control evidence, not a quality, load, public-network, or endurance claim.
|
||||
- The current quality stack uses one strict, non-persistent `QualitySettings` value with bounded ceilings and clarity/balanced/fluid preference. Sender readback and limitation warnings are manual diagnostics; ADR-0007's demand-driven `HIGH + optional LOW <= 2` controller is accepted but unimplemented.
|
||||
- Chrome 151 synthetic 720p30 propagated balanced/clarity to one host and three viewers without changing peer identities; all viewers kept decoding and fanout stayed host two/relay one. This proves control continuity only.
|
||||
- Per-frame encode/decode diagnostics use adjacent non-overlapping `getStats()` deltas rather than connection-lifetime averages; first, empty, changed-stream, and reset intervals remain unknown and rebase.
|
||||
- Release `769de201f7cc` passed main CI, repository hygiene, type checking, 22 Vitest files/274 tests, both builds, zero production dependency vulnerabilities, atomic activation, public health/access checks, exact bundle verification, and service/log checks.
|
||||
|
||||
@@ -65,11 +65,12 @@ These are measurement gates, not performance claims.
|
||||
- Project license and distribution model, which determines whether GPL/AGPL sources can move beyond study-only use.
|
||||
- Initial deployment regions and expected mainland China, Hong Kong, and overseas network mix.
|
||||
- Whether voice chat ever enters scope or Screener stays complementary to an existing voice application.
|
||||
- Exact evidence thresholds/windows and the native hardware matrix for ADR-0007; #28's minimum-of-two rule is not a candidate.
|
||||
|
||||
## Source Of Truth
|
||||
|
||||
- Documentation index and current phase: `docs/README.md` and `docs/status.md`
|
||||
- Requirements and design: `docs/需求理解.md` and `docs/方案设计.md`
|
||||
- Media research: `docs/research/webrtc-p2p-screen-sharing.md`, `docs/research/realtime-quality-adaptation.md`, `docs/research/browser-screen-audio-quality.md`, `docs/research/peer-assisted-media.md`, `docs/research/low-server-media-routes.md`, `docs/research/advanced-peer-distribution.md`, and `docs/research/native-shared-encode-sender.md`
|
||||
- Architecture: ADR-0001/0002, proposed ADR-0004/0005, and rejected/superseded ADR-0003
|
||||
- Architecture: ADR-0001/0002, proposed ADR-0004/0005, accepted ADR-0007, and rejected/superseded ADR-0003
|
||||
- Deployment and maintenance: `docs/deployment.md`, `docs/maintenance.md`, `AGENTS.md`, and `.codex/`
|
||||
@@ -49,12 +49,12 @@ synthetic source -> shared encoder coordinator -> encoder proxy A -> PC A -> bro
|
||||
the same reference-counted encoded buffer to both callbacks.
|
||||
- Reject the second edge when codec, profile, dimensions, or layer configuration
|
||||
differ. Never hide incompatibility by creating a second physical encoder.
|
||||
- In this fixed single-layer spike, aggregate the two `SetRates()` requests by
|
||||
taking the minimum total target bitrate and minimum requested frame rate,
|
||||
bounded by the selected profile. Do not attempt to merge per-edge loss, RTT,
|
||||
bandwidth-allocation, or multilayer state. A weak edge can therefore lower
|
||||
quality for the strong edge; independent quality requires another encode or
|
||||
a separately accepted layered design.
|
||||
- For this fixed single-layer spike only, aggregate the two `SetRates()`
|
||||
requests by taking the minimum total target bitrate and frame rate, bounded by
|
||||
the selected profile. This is an experiment safety policy, not a product
|
||||
design: one weak edge can lower a healthy edge, so productization is
|
||||
forbidden. Independent quality requires another representation, an accepted
|
||||
scalable design, or transcoding.
|
||||
- Coalesce PLI/keyframe requests so one physical keyframe is delivered to both
|
||||
packetizers. RTP sequence numbers, SSRC, TWCC, pacing, retransmission caches,
|
||||
RTCP, DTLS-SRTP, and ICE remain independent per edge.
|
||||
@@ -104,8 +104,9 @@ This certainty moves more responsibility into the application. Pion's default
|
||||
interceptors provide NACK, RTCP reports, statistics, and transport-wide feedback,
|
||||
but an external encoder still needs explicit send-side bandwidth estimation,
|
||||
aggregation of both edge targets, encoder bitrate control, PLI handling, and
|
||||
bounded RTP/RTX queues. A Pion route must use the same minimum-edge rate rule and
|
||||
must not invent a custom SRTP, ICE, or congestion protocol.
|
||||
bounded RTP/RTX queues. The already bounded #23/#28 ladder uses the same
|
||||
minimum-edge safety rule and must not invent a custom SRTP, ICE, or congestion
|
||||
protocol. A product route must follow ADR-0007 instead.
|
||||
|
||||
## Stacked Validation Ladder
|
||||
|
||||
@@ -140,6 +141,40 @@ sequence; leg 2 recorded no drop, NACK, or replay. Both viewers decoded another
|
||||
359 frames after the shared recovery marker. This is a go result only for the
|
||||
bounded live target plus primary-SSRC recovery candidate.
|
||||
|
||||
## Product Quality Direction
|
||||
|
||||
PR #28 establishes only that one live `VideoEncoder` object accepted one
|
||||
conservative feedback-derived reconfiguration while feeding two transports. It
|
||||
does not establish heterogeneous feedback behavior and must not evolve into a
|
||||
room-wide minimum controller.
|
||||
|
||||
ADR-0007 defines the product boundary: normally run one shared `HIGH` encoded
|
||||
representation; start one shared `LOW` only while correlated sender/viewer
|
||||
evidence proves that paths persistently cannot sustain `HIGH`; keep healthy
|
||||
paths on `HIGH`; and stop `LOW` after every weak path sustains the longer
|
||||
recovery window. The representation limit is two, never one per viewer. The
|
||||
independent host budget remains at most two downstream media edges.
|
||||
|
||||
If a qualified hardware/power-efficient second encoder is unavailable or its
|
||||
measured game load is unacceptable, `LOW` fails closed for weak paths while
|
||||
healthy paths keep `HIGH`.
|
||||
|
||||
Each path has only `HIGH` and `FALLBACK` state with asymmetric consecutive
|
||||
entry/recovery windows, not a composite score. Viewer requests are
|
||||
authenticated, rate-limited, deduplicated advice and need sender transport and
|
||||
encode evidence plus viewer receive/decode corroboration. UA and device
|
||||
identity do not participate in quality control.
|
||||
|
||||
A native relay forwards the selected encoded packets and must not decode or
|
||||
re-encode them. An ordinary non-scalable representation cannot be forwarded
|
||||
into a second quality; the bounded choices are a temporary second encode, SVC,
|
||||
or transcoding. The default is the temporary second representation. SVC is
|
||||
considered only for a later strict-one-output requirement and only after the
|
||||
exact codec/mode and a hardware or power-efficient implementation pass the game
|
||||
performance matrix; software fallback is rejected rather than hidden. Future
|
||||
dual-tree/striped distribution may reduce two-copy host upload toward one copy
|
||||
plus redundancy, but does not block dual-representation work.
|
||||
|
||||
## Product Stop Line
|
||||
|
||||
The stock Pion GCC plus negotiated RFC 4588 RTX composition is
|
||||
@@ -155,7 +190,9 @@ PLI/FIR-to-encoder control, heterogeneous downstream estimates, bounded burst
|
||||
and sustained loss, audio/A-V synchronization, mixed direct/TURN edges,
|
||||
reconnect isolation, browser diversity, lifecycle, and sustained CPU/GPU,
|
||||
memory, latency, quality, and upload measurements. A third host edge, custom
|
||||
RTP/SRTP, or a custom congestion-control framework remains out of scope.
|
||||
RTP/SRTP, or a custom congestion-control framework remains out of scope. The
|
||||
#28 minimum-of-two policy is also out of scope for product code; retaining its
|
||||
evidence does not retain its policy.
|
||||
|
||||
## Primary Sources And License Boundary
|
||||
|
||||
@@ -176,6 +213,9 @@ RTP/SRTP, or a custom congestion-control framework remains out of scope.
|
||||
- [Pion send-side bandwidth estimator](https://github.com/pion/interceptor/blob/main/pkg/gcc/send_side_bwe.go)
|
||||
- [Pion WebRTC license](https://github.com/pion/webrtc/blob/main/LICENSE)
|
||||
- [WebCodecs](https://www.w3.org/TR/webcodecs/)
|
||||
- [W3C WebRTC Statistics](https://www.w3.org/TR/webrtc-stats/)
|
||||
- [W3C WebRTC SVC](https://www.w3.org/TR/webrtc-svc/)
|
||||
- [W3C Media Capabilities](https://www.w3.org/TR/media-capabilities/)
|
||||
- [Pion WebRTC v4.2.18](https://github.com/pion/webrtc/tree/v4.2.18)
|
||||
- [Pion Interceptor v0.1.47](https://github.com/pion/interceptor/tree/v0.1.47)
|
||||
- [RFC 4585 RTP/AVPF feedback](https://www.rfc-editor.org/rfc/rfc4585.html)
|
||||
|
||||
@@ -37,12 +37,17 @@ described as automatically detecting game motion and forcing a 720p cap.
|
||||
|
||||
On 2026-08-19 the deployed build was reported to deliver very low resolution,
|
||||
bitrate, and frame rate across all three profiles. This is a user observation,
|
||||
not yet an instrumented result. The highest-confidence explanation is the
|
||||
combination of one independent peer connection, sender, encoder pipeline, and
|
||||
upload copy per viewer with `balanced` adaptation: host CPU or uplink pressure
|
||||
can grow per edge, and the browser may then reduce both dimensions. Peer/SFU
|
||||
code now ships in `769de201f7cc`, but its production configuration is absent;
|
||||
relay re-encoding and LiveKit therefore cannot explain that earlier observation.
|
||||
not yet an instrumented result. Draft PR #28's minimum-of-two feedback loop was
|
||||
never deployed and cannot have caused it. The production `getStats()` probes are
|
||||
read-only and do not trigger a downgrade. Peer/SFU code ships in `769de201f7cc`,
|
||||
but its production configuration is absent, so relay re-encoding and LiveKit
|
||||
cannot explain the earlier observation either.
|
||||
|
||||
Plausible causes remain: capture settings below the request; one independent
|
||||
encoder pipeline per peer exhausting CPU/GPU; GCC reacting to host uplink or a
|
||||
TURN path; treating `maxBitrate` as a target when it is only a ceiling; browser
|
||||
rewriting or scaling sender parameters; or receive loss, jitter, decode, and
|
||||
display scaling. None is selected as the root cause before correlated evidence.
|
||||
|
||||
One controlled capture should classify the problem before changing constants:
|
||||
|
||||
@@ -63,6 +68,81 @@ are ceilings rather than targets. Raising them cannot repair CPU or bandwidth
|
||||
limitation and should only follow evidence that the encoder is already pinned
|
||||
to the ceiling while spare transport capacity remains.
|
||||
|
||||
## Evidence Before Adaptation
|
||||
|
||||
Verified specification facts: the W3C stats model supports the needed
|
||||
separation but does not produce the product decision itself.
|
||||
`MediaStreamTrack.getSettings()` supplies the actual
|
||||
capture dimensions/frame rate. Outbound RTP exposes emitted dimensions/FPS,
|
||||
byte and retransmission counters, encode time, target bitrate when available,
|
||||
and the current `qualityLimitationReason`; the selected candidate pair can
|
||||
expose RTT and available outgoing bitrate. Inbound RTP exposes received
|
||||
dimensions/FPS, byte/loss/jitter counters, decoded/dropped frames, and freeze
|
||||
counters when implemented. Stats members may be absent, and cumulative values
|
||||
must be compared across two samples rather than treated as interval values.
|
||||
|
||||
Correlate one time interval and media generation across:
|
||||
|
||||
| Evidence | Fields |
|
||||
| --- | --- |
|
||||
| A. Host capture | actual width, height, FPS from `getSettings()` |
|
||||
| B. Host outbound | width/FPS/bitrate, target/available bitrate, interval encode time, limitation reason, path, RTT, loss/retransmission, and derived negotiated codec/profile/parameters plus applicable `scalabilityMode` |
|
||||
| C. Viewer inbound | width/FPS/bitrate, loss, jitter, interval decode/drop/freeze, corresponding derived codec/profile/parameters/layer, and actual decode behavior |
|
||||
|
||||
Product inference from those facts: use the following ordered classification:
|
||||
|
||||
| Correlated observation | Likely boundary |
|
||||
| --- | --- |
|
||||
| Capture low | capture or constraints |
|
||||
| Capture high; outbound low; `cpu` | encoder/resource pressure |
|
||||
| Capture high; outbound low; `bandwidth` | GCC, uplink, or TURN/path pressure |
|
||||
| Outbound healthy; inbound low | transport or receiver path |
|
||||
| Inbound healthy; image visibly blurry | bitrate/quantization, codec, or display scaling |
|
||||
|
||||
This is a diagnostic classification, not a weighted health score. Missing or
|
||||
reset evidence remains unknown and rebases the interval.
|
||||
|
||||
The smallest implementation sequence is local A+B correlation in one host
|
||||
sampling tick, including the interval, media/stat identity, and valid deltas.
|
||||
Only after that is trustworthy should a minimal authenticated C report carry
|
||||
the receive/decode and derived negotiation signals needed by the two-state
|
||||
predicate. It never carries raw SDP, raw stats, candidate addresses, or other
|
||||
identifiers; a general remote stats stream or telemetry pipeline is unnecessary.
|
||||
|
||||
## Accepted Adaptation Direction
|
||||
|
||||
ADR-0007 rejects room-wide worst-link adaptation. Healthy operation has one
|
||||
shared `HIGH` representation. A viewing path enters `FALLBACK` only after
|
||||
multiple consecutive windows show insufficient bandwidth, freezes, or decode
|
||||
pressure in correlated sender/viewer evidence. The first verified weak path
|
||||
starts one shared `LOW`; all weak paths reuse it while healthy paths remain on
|
||||
`HIGH`. Recovery requires a longer stable window than entry. When the last weak
|
||||
path recovers, stop `LOW`. The hard representation limit is two, never one per
|
||||
viewer.
|
||||
|
||||
`LOW` itself is conditional: if no qualified hardware/power-efficient encoder
|
||||
path exists or the second encoder exceeds the measured CPU/GPU/game budget, the
|
||||
controller preserves `HIGH` and fails visibly for the weak path. It never buys
|
||||
weak-path recovery by degrading healthy paths.
|
||||
|
||||
A viewer's `LOW` request is advisory and must be authenticated, session-bound,
|
||||
rate-limited, deduplicated, and corroborated by sender transport/encode and
|
||||
viewer receive/decode stats. UA or device-model detection is not quality
|
||||
evidence; the current mobile/iPad heuristic remains restricted to relay
|
||||
capacity.
|
||||
|
||||
An ordinary non-scalable stream cannot yield a second independent quality by
|
||||
packet forwarding alone. The alternatives are a second representation,
|
||||
scalable layers, or relay/SFU transcoding. Screener chooses the temporary second
|
||||
representation first. SVC is conditional on a future strict-one-output need,
|
||||
an exact negotiated mode, a positively established hardware or power-efficient
|
||||
path, and measured game performance. WebRTC-SVC permits the browser to return a
|
||||
different configured `scalabilityMode`; Media Capabilities reports support and
|
||||
expected smoothness/power efficiency for a specified configuration; WebCodecs
|
||||
defines `hardwareAcceleration` only as a hint the user agent may ignore.
|
||||
Therefore none is, by itself, proof of a particular hardware encoder, and a
|
||||
software SVC fallback must not be silent.
|
||||
|
||||
## Why Offline Encoding Presets Do Not Transfer
|
||||
|
||||
[Maruko Toolbox](https://github.com/wzxjohn/marukotoolbox) is an Apache-2.0
|
||||
@@ -122,9 +202,9 @@ resolution, frame rate, or bitrate.
|
||||
- Pausing the picture disables the existing video track, producing black video
|
||||
without closing the room or media connection. Audio remains enabled.
|
||||
|
||||
The implementation deliberately stops at manual bounded controls. It adds no
|
||||
composite score, periodic adjustment, codec forcing, SDP bitrate manipulation,
|
||||
or scene detector. Three consecutive samples of one non-`none` native
|
||||
The deployed implementation deliberately stops at manual bounded controls. It
|
||||
adds no composite score, periodic adjustment, codec forcing, SDP bitrate
|
||||
manipulation, or scene detector. Three consecutive samples of one non-`none` native
|
||||
`qualityLimitationReason` produce one explanatory warning; a reason change or
|
||||
recovery resets it and never triggers a media action.
|
||||
|
||||
@@ -150,15 +230,18 @@ CPU/GPU cost, public networks, or sustained behavior.
|
||||
preference identically.
|
||||
- No automatic composite quality score or periodic profile controller before
|
||||
the controlled production capture identifies a reproducible bottleneck.
|
||||
- No room-wide minimum-of-viewers target, per-viewer encoder, UA/device quality
|
||||
ranking, or silent software SVC fallback.
|
||||
|
||||
## Verification Gate
|
||||
|
||||
Use the same static UI scene and deterministic high-motion game scene at
|
||||
720p30, 1080p30, and 1080p60. Record actual capture and outbound dimensions/fps,
|
||||
bitrate, `qualityLimitationReason`, codec, encoder implementation, encode time,
|
||||
dropped frames, RTT, packet loss, and host CPU/GPU utilization. A live profile
|
||||
change must preserve peer connection IDs, avoid a second source prompt, and
|
||||
visibly converge to the requested bounds.
|
||||
720p30, 1080p30, and 1080p60. Correlate A capture, B outbound, and C inbound at
|
||||
the same interval. Record actual dimensions/FPS/bitrate, limitation reason,
|
||||
codec, encoder implementation, interval encode/decode cost, drops/freezes,
|
||||
jitter, RTT, loss/retransmission, selected direct/TURN path, and host CPU/GPU.
|
||||
A live profile change must preserve peer connection IDs, avoid a second source
|
||||
prompt, and visibly converge to the requested bounds.
|
||||
|
||||
Compute per-frame encode and decode cost from adjacent samples of cumulative
|
||||
`totalEncodeTime`/`totalDecodeTime` and frame counters. The first sample, a
|
||||
@@ -172,7 +255,9 @@ pipeline.
|
||||
Compare image readability and motion continuity instead of declaring success
|
||||
from FPS alone. If reproducible evidence later shows that `balanced` still
|
||||
oscillates or makes the wrong tradeoff on supported machines, revise the fixed
|
||||
profiles before adding an application-level adaptation controller.
|
||||
profiles before enabling ADR-0007. Its acceptance matrix must prove one healthy
|
||||
viewer remains `HIGH` while another enters/recover from `FALLBACK`, that all weak
|
||||
viewers share at most one `LOW`, and that `LOW` stops after sustained recovery.
|
||||
|
||||
## Primary Sources
|
||||
|
||||
@@ -180,6 +265,9 @@ profiles before adding an application-level adaptation controller.
|
||||
- [W3C Screen Capture](https://www.w3.org/TR/screen-capture/)
|
||||
- [W3C WebRTC](https://www.w3.org/TR/webrtc/)
|
||||
- [W3C WebRTC Statistics](https://www.w3.org/TR/webrtc-stats/)
|
||||
- [W3C WebRTC SVC](https://www.w3.org/TR/webrtc-svc/)
|
||||
- [W3C Media Capabilities](https://www.w3.org/TR/media-capabilities/)
|
||||
- [W3C WebCodecs](https://www.w3.org/TR/webcodecs/)
|
||||
- [MDN `RTCRtpSender.setParameters()`](https://developer.mozilla.org/en-US/docs/Web/API/RTCRtpSender/setParameters)
|
||||
- [Chromium `motion` to libwebrtc `kFluid` bridge](https://chromium.googlesource.com/chromium/src/third_party/+/refs/heads/main/blink/renderer/modules/peerconnection/media_stream_video_webrtc_sink.cc)
|
||||
- [libwebrtc `motion`/`kFluid` sender classification](https://webrtc.googlesource.com/src/+/3b1eab8a69cb5078befb021c5492d3f204a7d6a2/pc/rtp_sender.cc)
|
||||
|
||||
+11
-8
@@ -18,19 +18,19 @@ deployed binary still uses one host peer connection per viewer and P2P/TURN.
|
||||
- Hidden routing is `direct P2P -> peer-assisted -> optional SFU`. `PEER_ASSISTED_ROOM_IDS` canaries exact rooms; others stay legacy P2P. Empty or missing means all, production is disabled, and the 22-file/284-test full suite covers both paths.
|
||||
- Complete fallback configuration adds a non-secret standby URL to peer-assisted authentication. Host and viewers import the SDK and make one token-free DNS/TLS warmup; no configuration means no field, import, request, participant, or media edge.
|
||||
- A viewer starts as a leaf each session and explicitly advertises relay capacity zero or one; the Web client reports detected mobile/iPad clients as zero and desktop-class browsers as one. Withdrawal stops future assignment without moving a healthy edge. Browser relays remain one-child; the host remains two-child.
|
||||
- Production uses one strict, memory-only video quality setting with bounded manual ceilings and sender readback. It also carries equal idle-stage share/join actions and default-closed technical details. Browser audio remains request/presence-only.
|
||||
- Production uses one strict, memory-only video quality setting with bounded manual ceilings and sender readback. ADR-0007's per-path `HIGH`/`FALLBACK` and demand-driven shared `LOW` are accepted design, not deployed behavior. Browser audio remains request/presence-only.
|
||||
|
||||
## Verified Evidence
|
||||
|
||||
- Release `769de201f7cc` passed main CI, repository hygiene, type checking, 22 Vitest files/274 tests, both builds, and zero production dependency vulnerabilities.
|
||||
- A short Chrome 151 synthetic `1/3/5/8` benchmark passed every topology check: host active edges peaked at two, relay edges at one, every viewer kept increasing decoded frames through the measurement window, and the slowest first decoded frame was about 1.05 seconds. Closing a first-level relay in the three-viewer run recovered in about 5.32 seconds without exceeding host fanout two.
|
||||
- A localhost/headless Chrome 151 run with one host, three viewers, and synthetic 720p30 propagated balanced and clarity settings to every participant. Every active sender matched preference readback, peer fingerprints stayed stable, all viewers decoded/rendered new frames, and fanout stayed host two/relay one. This proves control continuity, not visual quality or performance.
|
||||
- Corrected Chrome 151/LiveKit 1.13.5 localhost/headless/video-only same-leaf A/B measured the cold path at 1.481 seconds to active and 2.257 seconds to render. Standby prepare arrived in 5 ms, active in 200 ms, and the same leaf rendered in 319.7 ms with 31 new decoded/frame-callback frames; 25 ms sampling kept host edge peak two. The roughly 86% result combines early SDK download/parse with token-free DNS/TLS/HTTP prewarm, creates no participant/media edge, and does not predict public-network performance.
|
||||
- Chrome 151 synthetic 720p30 propagated balanced/clarity to one host and three viewers without changing peer identities; all kept decoding and fanout stayed host two/relay one. This proves control continuity only.
|
||||
- Chrome 151/LiveKit localhost A/B measured cold active/render at 1.481/2.257 seconds and standby at 0.200/0.320 seconds while host edges stayed at two. This video-only local result includes SDK startup plus prewarm and does not predict public networks.
|
||||
- Atomic production activation passed public/local health, exact served-bundle hash, unauthenticated 401, room `1` page, SQLite preservation, active Screener/nginx/coturn, and a clean warning log. The old release remains available for rollback.
|
||||
- Chrome 151 CDP checks at 320/375/390 CSS px keep the two idle-stage actions equal, on one row, 44 px high, and free of horizontal overflow. The Viewer waiting page also has no overflow; its details checkbox starts false, changes locally, and resets after navigation. These checks cover idle/waiting states, not live media.
|
||||
- Chrome 151 checks at 320/375/390 CSS px passed equal 44 px entry actions, waiting-page overflow, and default-closed/resetting details. Live media was not covered.
|
||||
- Local diagnostics use adjacent non-overlapping `getStats()` deltas; empty, changed-stream, and reset intervals rebase instead of publishing lifetime averages.
|
||||
- Production HTTPS/WSS, access cookie, room/WebSocket authorization, certificate renewal, public STUN, and authenticated TURN/UDP and TURN/TCP relay-only bidirectional paths are verified. TURN/TLS is intentionally disabled.
|
||||
- Draft native ladder #16/#18/#22/#23/#25/#28 passed a bounded 720p30 two-leg loop: one WebCodecs object accepted the lower stock-GCC target, one isolated loss recovered, and leg 2 stayed clean. Research-only.
|
||||
- Draft native ladder #16/#18/#22/#23/#25/#28 passed a bounded 720p30 two-leg loop: one WebCodecs object accepted a conservative minimum stock-GCC target, one isolated loss recovered, and leg 2 stayed clean. That minimum rule is experiment-only and prohibited from productization.
|
||||
|
||||
## Unverified Boundaries
|
||||
|
||||
@@ -39,13 +39,15 @@ deployed binary still uses one host peer connection per viewer and P2P/TURN.
|
||||
- Silent partitions can wait 30 to 60 seconds for heartbeat detection before the default 5-second grace; this remains unverified.
|
||||
- Real screen/game audio, heterogeneous machines/networks, mobile lifecycle, the new production quality/pause cycle, room `1` stop/re-publish, and sustained profiles remain unverified.
|
||||
- Earlier production showed severe degradation across all profiles. The new controls permit clarity/balanced comparison, but a controlled 1/2/3-viewer and TURN sample must still isolate capture, CPU, path, and receiver limits before any quality claim or automatic controller.
|
||||
- ADR-0007 is unimplemented. First correlate same-tick capture plus outbound window/identity/deltas; only then add a minimal authenticated viewer receive/decode report. Runtime evidence, not UA/device identity, must drive `HIGH`/`FALLBACK`; representations stay at `HIGH + optional LOW <= 2`.
|
||||
- Browser relays re-encode. The native ladder proves one WebCodecs object, not one physical/hardware encode. Stock GCC plus RTX is no-go; no-RTX passed one controlled loss but weakens statistics. Audio, heterogeneous estimates, broader loss, direct/TURN, reconnect, browser diversity, product wiring, striping, and multi-parent assembly remain unverified.
|
||||
- The endpoint budget remains host/relay at most two downstream edges; the browser path is host two/viewer one.
|
||||
- The corrected standby smoke is localhost/headless/video-only; public DNS/TLS reuse, transport, audio, shaping, load, mobile, endurance, and sub-25 ms overlap remain open.
|
||||
|
||||
## Next Milestone
|
||||
|
||||
First validate the deployed quality controls on real game capture. Before any
|
||||
First validate the deployed quality controls on real game capture and implement
|
||||
the bounded A+B probe described above. Before any
|
||||
broad route enablement, use one exact-room canary and validate standby gains
|
||||
across public transports, rollback, reconnect, edge counts, egress, and load.
|
||||
|
||||
@@ -59,8 +61,9 @@ picture, about 8 seconds total. Silent partitions include their detection delay.
|
||||
ADR-0004 fails closed. Native shared encode and packet/layer striping remain
|
||||
separate experiments, not ways to relabel a failed browser route.
|
||||
|
||||
The native ladder stops at #28. Do not bypass stock GCC/RTX with custom
|
||||
transport/control; no-RTX remains research-only.
|
||||
The native ladder stops at #28. Its minimum-of-two feedback policy is not a
|
||||
product candidate. Do not bypass stock GCC/RTX with custom transport/control;
|
||||
no-RTX remains research-only.
|
||||
|
||||
## Blockers And Decisions
|
||||
|
||||
|
||||
+26
-3
@@ -133,9 +133,9 @@ Zod 在服务边界拒绝未知类型、超长字符串、非法 SDP/candidate
|
||||
| 1080p30 | 1920x1080、30 fps | 5 Mbps/peer |
|
||||
| 720p30 | 1280x720、30 fps | 3 Mbps/peer |
|
||||
|
||||
浏览器可返回更低设置,界面以 `track.getSettings()` 显示实际捕获结果。视频 track 保留 `contentHint = "motion"`;待合并质量控制默认使用 `degradationPreference = "maintain-resolution"`,三个推荐档位都采用清晰优先。折叠高级区将设置严格限制为 720p/1080p/1440p、15 至 60 整数 FPS、2 至 12 Mbps,以及 `maintain-resolution`、`balanced`、`maintain-framerate` 三种偏好。它们只是 ceiling 和浏览器偏好,不保证实际分辨率、帧率或码率。
|
||||
浏览器可返回更低设置,界面以 `track.getSettings()` 显示实际捕获结果。生产 release `769de201f7cc` 已部署有界手动质量控制:视频 track 保留 `contentHint = "motion"`,默认使用清晰优先的 `degradationPreference = "maintain-resolution"`,并允许显式选择 balanced 或 fluid。折叠高级区将设置严格限制为 720p/1080p/1440p、15 至 60 整数 FPS、2 至 12 Mbps,以及 `maintain-resolution`、`balanced`、`maintain-framerate` 三种偏好。它们只是 ceiling 和浏览器偏好,不保证实际分辨率、帧率或码率。
|
||||
|
||||
每次 `setParameters()` 都必须从当次 `getParameters()` 派生更新,然后读回 requested/applied `maxBitrate`、`maxFramerate`、`scaleResolutionDownBy` 和 degradation preference。拒绝或浏览器改写必须进入可见状态,不能只写 `console.warn`。生产版本仍使用 `balanced`,用户报告的严重降级尚未完成受控分类;本改动只提供有界 clarity/balanced 控制,不得据此宣称画质提升。下一次真实质量验收仍需在同一高动态场景采样 1、2、3 名观看者和 TURN 路径。不得只提高 ceiling、强制 codec 或修改 SDP。
|
||||
每次 `setParameters()` 都必须从当次 `getParameters()` 派生更新,然后读回 requested/applied `maxBitrate`、`maxFramerate`、`scaleResolutionDownBy` 和 degradation preference。拒绝或浏览器改写必须进入可见状态,不能只写 `console.warn`。线上已是 clarity-first bounded controls,但用户报告的严重降级尚未完成受控分类,ADR-0007 自动质量控制也尚未实现;不得据此宣称画质提升。下一次真实质量验收仍需在同一高动态场景采样 1、2、3 名观看者和 TURN 路径。不得只提高 ceiling、强制 codec 或修改 SDP。
|
||||
|
||||
PoC 不强制 codec,也不做 SDP munging。先通过 stats 记录实际协商 codec、`encoderImplementation` 和 `powerEfficientEncoder`,再依据目标机器测量决定是否优先 H.264。这样避免静态 codec 偏好意外触发软件编码。
|
||||
|
||||
@@ -149,7 +149,27 @@ PoC 不强制 codec,也不做 SDP munging。先通过 stats 记录实际协商
|
||||
|
||||
`frameRate` 约束和 sender 的 `maxFramerate` 都是目标或上限;浏览器仍控制实际捕获、重复帧处理、编码和拥塞降级。静态画面可能由浏览器减少实际编码数据,但标准没有保证每个实现以同一种方式节省采集或 GPU 开销。本 PoC 不添加画面差分、定时改约束或自定义动态 FPS 状态机,先用 `framesEncoded`、实际发送码率、`totalEncodeTime`、质量受限原因和主机资源占用测量真实问题。
|
||||
|
||||
自动全房间降档同样需要可信的聚合策略,放在测量后的下一阶段;本 PoC 提供手动档位和明确的受限原因,不伪装成已经解决自适应问题。
|
||||
自动质量控制采用 ADR-0007 的按需双表示,而不是全房间按最差链路降档。正常状态只运行一套共享 `HIGH` 编码;第一条被真实统计持续证明无法承受的观看路径出现时,临时启动一套共享 `LOW`,所有弱路径复用它,健康路径继续接收 `HIGH`。最后一条弱路径经过更长的稳定恢复窗口回到 `HIGH` 后关闭 `LOW`。表示数量硬限制为 `HIGH + optional LOW <= 2`,不能按 viewer 增长为 N 套编码。若没有合格硬件/power-efficient 编码路径或第二编码实测破坏游戏 CPU/GPU 预算,则不启动 `LOW`,保留健康路径的 `HIGH` 并让弱路径明确降级或失败。
|
||||
|
||||
每条观看路径只维护 `HIGH`/`FALLBACK` 两态。`HIGH -> FALLBACK` 需要多个连续采样窗口中的带宽不足、冻结或解码跟不上;`FALLBACK -> HIGH` 使用不同门槛和更长的稳定窗口,避免来回抖动。不把 RTT、CPU、码率、设备信息等压成一个综合分数。具体窗口和阈值必须由真实质量矩阵确定,而不是先写死一个猜测。
|
||||
|
||||
状态决策必须关联同一时间区间和媒体代次的三段证据:
|
||||
|
||||
| 阶段 | 必要证据 |
|
||||
| --- | --- |
|
||||
| A. Host capture | `track.getSettings()` 的实际宽、高、FPS |
|
||||
| B. Host outbound | 实际宽高/FPS/码率、可用或目标码率、区间 encode time、`qualityLimitationReason`、RTT、丢包、重传、direct/TURN 路径,以及派生后的实际协商视频 codec/profile/parameters 和适用时的 `scalabilityMode` |
|
||||
| C. Viewer inbound | 实际宽高/FPS/码率、丢包、jitter、区间 decoded/dropped frame 和 freeze,以及对应派生 codec/profile/parameters、适用时的 `scalabilityMode` 和实际解码表现 |
|
||||
|
||||
判断顺序固定为:Capture 低归为采集或约束;Capture 高而 outbound 低且 reason=`cpu` 归为编码资源;Capture 高而 outbound 低且 reason=`bandwidth` 归为 GCC、上行或 TURN/路径;outbound 正常而 inbound 低归为传输或接收;inbound 指标正常但肉眼模糊归为码率/量化、协商 codec/profile/parameters、适用的 scalable layer 或显示缩放。B/C 必须关联同一派生协商结果与实际解码表现。累计统计只计算相邻、不重叠区间的 delta;缺字段、stats ID/媒体代次变化或计数器回退时重新建立基线,不能把未知当零。
|
||||
|
||||
探针按最小顺序落地:先在 host 同一个采样 tick 内关联 A+B,并携带明确 window、媒体/stats identity 和有效 delta;验证可靠后才增加只包含两态判定所需字段的受限、已鉴权 C 上报。不建设通用遥测 schema,只传派生后的协商字段,绝不上传 raw stats、raw SDP、candidate 地址或其他标识符。
|
||||
|
||||
Viewer 可通过已鉴权、绑定当前房间/路径代次的控制面请求 `LOW`,但请求只是一条观测输入,必须限频和去重,并与 sender 的 Transport-CC/GCC、实际码率、RTT、丢包、重传及编码限制、派生协商 codec/profile/parameters(适用时含 `scalabilityMode`)和 viewer 的实际解码/丢帧/冻结表现交叉验证。UA、平台和设备型号不参与质量决策;现有 mobile/iPad 检测只用于保守设置 relay capacity。
|
||||
|
||||
若以后要求任何时刻严格只有一个编码输出,才单独评估 SVC:每条连接可独立选择基础层或增强层,但必须读回实际 codec/`scalabilityMode`,确认目标硬件或 power-efficient 编码路径,并通过游戏性能和亚秒延迟验收。浏览器的硬件偏好不是保证,不能静默回退为软件 SVC;常驻 SVC 不是默认方案。普通非分层编码包不能只靠转发生成另一档画质,实现路径差异只能用第二表示、SVC 或 relay/SFU 转码,当前优先第二表示。
|
||||
|
||||
质量控制不放宽媒体拓扑预算:host 仍最多两条下游媒体边;native relay 只转发选定表示的编码包,不解码重编码。后续双树或条带化可把 host 上传从约两份完整码率推向一份加必要冗余,但不阻塞按需双表示的实现。
|
||||
|
||||
## TURN 配置与轮换
|
||||
|
||||
@@ -287,6 +307,9 @@ spike 复用现有 screen stream 和浏览器 codec 协商,覆盖当前桌面
|
||||
- [Media Capture Content Hints](https://www.w3.org/TR/mst-content-hint/)
|
||||
- [RTCRtpSender.replaceTrack](https://www.w3.org/TR/webrtc/#dom-rtcrtpsender-replacetrack)
|
||||
- [WebRTC Stats](https://www.w3.org/TR/webrtc-stats/)
|
||||
- [WebRTC SVC](https://www.w3.org/TR/webrtc-svc/)
|
||||
- [Media Capabilities](https://www.w3.org/TR/media-capabilities/)
|
||||
- [WebCodecs](https://www.w3.org/TR/webcodecs/)
|
||||
- [WebRTC official samples](https://webrtc.github.io/samples/)
|
||||
- [coturn TURN server documentation](https://github.com/coturn/coturn/blob/master/README.turnserver)
|
||||
- [coturn example configuration](https://github.com/coturn/coturn/blob/master/examples/etc/turnserver.conf)
|
||||
|
||||
+10
-2
@@ -21,7 +21,9 @@
|
||||
- [ ] 分享端 fanout 硬上限:分享端任何时刻最多维护两条下游媒体边,包括加入、断线、换父和恢复过程。第三名及后续观看者由系统自动分配给通过验收的 peer-assisted 拓扑,必要时自动回退到部署已提供的可选 SFU;两者都不可用时才明确拒绝,不能隐藏地回退为分享端第三条连接。
|
||||
- [ ] P2P 优先:每条逻辑媒体边的 ICE 必须先尝试直接 UDP 路径,服务器不得默认中转可直连的媒体。一个或两个观看者优先直接连接分享端;更多观看者的 peer-assisted 行为须先通过 ADR-0004 的实验门槛。
|
||||
- [ ] NAT 兜底:每条 parent-child 边独立进行 ICE;生产环境必须提供 STUN、TURN/UDP 与 TURN/TCP candidates,通过优先级偏好直连和 UDP,同时允许无法使用 UDP 的网络选择 TCP,不能因为某条边的网络与其他边不同而静默失败。
|
||||
- [ ] 质量控制:保留 1080p60(8 Mbps)、1080p30(5 Mbps)、720p30(3 Mbps)三档推荐上限,默认采用清晰优先;分享期间可直接换档或应用有界高级设置,不重新选择来源或重建健康连接。Peer-assisted relay 与可选 SFU 必须接收同一个严格校验的房间级权威设置,覆盖当前和以后建立的 sender;普通 P2P 信令结构保持不变。任何设置都只是浏览器上限和偏好,必须读回实际 sender 参数并显示参数拒绝、改写和持续受限原因。自定义动态 FPS、综合评分或周期调档只在统计证明必要后实现。
|
||||
- [ ] 质量控制:保留 1080p60(8 Mbps)、1080p30(5 Mbps)、720p30(3 Mbps)三档推荐上限,默认采用清晰优先;分享期间可直接换档或应用有界高级设置,不重新选择来源或重建健康连接。Peer-assisted relay 与可选 SFU 必须接收同一个严格校验的房间级权威设置,覆盖当前和以后建立的 sender;普通 P2P 信令结构保持不变。任何设置都只是浏览器上限和偏好,必须读回实际 sender 参数并显示参数拒绝、改写和持续受限原因。不做自定义动态 FPS、综合评分或周期性全房间调档。
|
||||
- [ ] 按需双表示:健康房间只运行一套共享 `HIGH` 编码;只有一个或多个观看路径被同一时段的 sender 与 viewer 统计持续证明无法承受时,才临时启动一套由全部弱路径共用的 `LOW` 编码。健康路径继续使用 `HIGH`,全部弱路径稳定恢复后关闭 `LOW`。编码表示硬上限为二,禁止按观看者增长为 N 套,也禁止把最差路径的目标应用到整个房间。没有合格硬件/power-efficient 路径或第二编码造成不可接受的 CPU/GPU/游戏负载时必须 fail closed:保护 `HIGH`,向弱路径明确显示降级或不可用。
|
||||
- [ ] 路径级质量状态:每条观看路径只使用 `HIGH` 与 `FALLBACK` 两态。进入 `FALLBACK` 需要多个连续窗口的带宽不足、冻结或解码跟不上证据,回到 `HIGH` 需要更长的稳定窗口;进入与退出使用不同门槛且不计算综合分数。Viewer 的低档请求必须经当前会话鉴权、限频和去重,并由 sender 的 GCC/Transport-CC、实际码率、RTT、丢包、重传、编码限制、派生后的实际协商视频 codec/profile/parameters(适用时含 `scalabilityMode`)与 viewer 的实际解码、丢帧、冻结证据交叉验证,不能仅凭自报生效;上报不得包含 raw SDP、candidate 地址或其他标识符。
|
||||
- [ ] 断线恢复:临时网络变化时尝试 ICE restart 和重连,并向用户显示当前连接状态。
|
||||
- [ ] 可观测性:采集候选类型、RTT、丢包、抖动、实际码率、帧率、编码耗时和质量受限原因。
|
||||
- [ ] 渐进式诊断:分享者和观看者默认只看到房间、画面、核心状态和常用控制;候选路径、RTT、丢包、编解码器、编码耗时等技术详情由一个明确的“显示连接详情”开关展开。隐藏诊断不能隐藏失败原因、音频缺失、持续画质受限或需要用户操作的告警。
|
||||
@@ -36,6 +38,7 @@
|
||||
- [ ] Peer-assisted 媒体实验:分享端最多两个 child、每名 viewer 最多一个 child;信令服务只按深度和服务端加入序号确定性地形成两条 sticky 平衡链,不加入综合评分或主动重平衡。每个 viewer 将收到的标准 remote track 通过新的 `RTCPeerConnection` 转发,因此会逐跳解码和重编码;必须测量而不能称为 shared encode。服务端只在受房间数限制的内存状态中保存当前严格校验的质量设置,不写入 SQLite;host 鉴权后先重报设置,在线 viewers 接收更新,后加入者从鉴权快照取得当前值。该功能在 ADR-0004 的实验门槛通过并由后续 ADR 正式接受前不得成为产品路径。
|
||||
- [ ] 原生 shared encode 分享端:后续独立的 Electron/原生分享端阶段必须保证一份兼容编码输出扇出到最多两套独立 WebRTC packetizer,并由单独 ADR 和测量约束。它降低分享端重复编码负载,但不消除每条出边各自的上传带宽。浏览器不保证跨 `RTCPeerConnection` 复用编码器,因此当前 Web spike 不实现、也不能宣称满足 shared encode 保证。
|
||||
- [ ] 原生 shared encode relay:具备原生 relay 能力的 audience 同样最多维护两条下游媒体边,并应让一份兼容编码输出供两条边复用。普通浏览器 audience 继续作为 leaf 或使用当前会逐跳重编码的实验路径,不伪称 shared encode。
|
||||
- [ ] 条件式 SVC:仅当后续需求要求任何时刻严格只有一个编码输出时才评估可伸缩编码,让各连接独立选择基础层或增强层。只有精确 codec/`scalabilityMode` 协商、硬件或 power-efficient 路径和游戏性能都实测通过时才允许启用;不得静默回退为高负载软件 SVC,常驻 SVC 不是默认方案。
|
||||
- [ ] 分包/分层多树实验:把编码包、对象或可独立解码层拆到多个 peer 路径以分散上传,只能在独立 spike 中验证。必须保持亚秒级,处理多父重组、排序、丢包恢复、关键帧、同步和节点退出;未通过门槛前不进入产品路由控制器。
|
||||
- [ ] 可选 SFU:部署者可选择是否提供单节点 SFU 资源;配置后由 `main` 中 PR #17 的路由控制器把它作为 P2P 与 peer-assisted 之后的自动兜底,而不要求用户选模式。它保持默认关闭、可逆,并承担每名观看者的服务器出口成本。已关闭 PR #12 的显式整房 SFU 模式已被取代,不是当前选项;自动 fallback 仍须通过 ADR-0005 门槛才可部署。
|
||||
- [ ] 分享端桌面封装:用 Electron 固定 Chromium 版本、改善选源/热键/系统音频体验,同时保留 Web 观看端。
|
||||
@@ -103,11 +106,16 @@
|
||||
- Given peer-assisted host 鉴权或直播中连续修改质量设置,When 服务端校验其分辨率、FPS、码率和偏好,Then 只有 host 的完整有界设置被接受,last-wins 地应用到当前和以后建立的 relay child sender 以及启用的 SFU publisher;越界、缺字段和多余字段被拒绝,普通 P2P 的鉴权 wire 不新增该字段且不接受同步消息。
|
||||
- Given 标准浏览器把收到的 remote track 直接加入下游 `RTCPeerConnection`,When 检查 relay 统计和 libwebrtc 路径,Then 明确记录新的 outbound encoder、编码耗时和 CPU/GPU 限制,不把它当作 encoded forwarding 或 shared encode 成果。
|
||||
- Given ADR-0004 的任一硬门槛只有在引入 Encoded Transform/DataChannel/WebCodecs 媒体、自定义拥塞控制、FEC/RTX、多树、综合评分、转码、codec ladder、浏览器 RTP 注入 hack 或放宽 host fanout 后才能满足,When 结束 bounded spike,Then 放弃并删除浏览器 relay 实验运行时代码,保留标准 P2P 和可选 SFU;后续 native shared encode sender 仍按独立 ADR 推进,但不能据此把失败的 relay 拓扑判为通过。
|
||||
- Given 新观看者会使预计总上行超过安全预算,When 其请求加入,Then 系统自动降低档位、拒绝该档位或建议改用外部直播服务,而不是持续堆积发送队列。
|
||||
- Given 新观看者会使预计总上行超过安全预算,When 其请求加入,Then 系统先遵守两边 fanout 预算选择 peer-assisted 或已配置的 SFU;没有合格路线时明确等待或拒绝,而不是创建第三条 host 边或持续堆积发送队列。质量不足只对经证据确认的观看路径使用共享 `LOW`,不能降低健康路径。
|
||||
|
||||
### 质量和恢复
|
||||
|
||||
- Given 用户选择任一推荐档位或高级质量参数,When 接收端实际分辨率、帧率或码率持续显著低于该目标,Then 诊断数据能区分采集约束、发送编码、带宽/拥塞、TURN 路径和接收端限制;产品不得只改变档位标签却继续隐藏实际降级。
|
||||
- Given 房间所有观看路径持续健康,When 自动质量控制运行,Then 只存在一套共享 `HIGH` 表示,且任何单路径短暂抖动或未经验证的低档请求都不启动第二套编码或降低其他路径。
|
||||
- Given 一个或多个路径在连续窗口中被 sender 与 viewer 证据共同确认无法承受 `HIGH`,When 它们进入 `FALLBACK`,Then 系统最多启动一套共享 `LOW`,只切换这些弱路径,其他路径继续 `HIGH`;所有弱路径经过更长稳定恢复窗口后返回 `HIGH` 并关闭 `LOW`。
|
||||
- Given 启动第二套编码会超过游戏 CPU/GPU 预算或没有合格硬件/power-efficient 路径,When 弱路径请求 `LOW`,Then 系统保留健康路径的 `HIGH` 并向弱路径明确显示降级或不可用,不得退回全房间最低目标或静默软件编码。
|
||||
- Given 同一时刻采集、host outbound 与 viewer inbound 证据,When 进行根因分类,Then 关联派生后的实际协商视频 codec/profile/parameters、适用时的 `scalabilityMode` 与真实解码表现,依次区分 Capture 低、Capture 高但 outbound 因 CPU 低、Capture 高但 outbound 因 bandwidth 低、outbound 正常但 inbound 低、以及 inbound 正常但肉眼模糊五类;不用 UA 或设备型号代替运行时证据,也不上传 raw SDP。
|
||||
- Given Viewer 发送低档请求,When 请求未鉴权、过频、重复、过期或没有匹配的 sender/viewer 统计,Then 请求不改变表示或路径状态;有效请求也只能作为交叉验证输入。
|
||||
- Given 受控 RTT/丢包测试条件,When 连续分享高动态游戏画面 30 分钟,Then 延迟、帧率与丢帧达到上述目标,且没有不可恢复的冻结。
|
||||
- Given 分享过程中切换 Wi-Fi/有线网络,When ICE 连接失败,Then 客户端触发 ICE restart,并在成功或超时后给出明确状态。
|
||||
- Given 浏览器未返回系统音轨,When 分享开始,Then 分享者在启播前看到“当前来源没有可共享音频”的提示。
|
||||
|
||||
Reference in new issue
Block a user