docs(media): define browser audio quality boundary (#24)

docs(media): define browser audio quality boundary
This commit is contained in:
AAA authored and GitHub committed 2026-08-19 15:27:15 +08:00
commit 8f7322e433
10 files changed
+204 -23

No files matched your search

+1
View File
@@ -12,6 +12,7 @@
- [Advanced peer distribution](./research/advanced-peer-distribution.md): multi-tree/SVC, encoded-object relay, native RTP forwarding, FEC/network-coding, MoQ, and measurable go/no-go gates.
- [Native shared-encode sender](./research/native-shared-encode-sender.md): bounded libwebrtc encoder-proxy risk spike, Draft PR #16's transport-only Pion fanout oracle, and hard failure gates.
- [Realtime quality adaptation](./research/realtime-quality-adaptation.md): browser degradation policy, live profile changes, offline-encoding boundaries, and measurement gates.
- [Browser screen-audio quality](./research/browser-screen-audio-quality.md): capture/source compatibility, standard readback limits, Opus negotiation boundaries, and the retained no-control decision.
- [Agent context governance research](./research/agent-context-governance.md): official Codex, Claude Code, Hermes Agent, and GitHub practices adopted by this repository.
- [ADR-0001](./adr/0001-p2p-first-media-topology.md): accepted P2P-first media topology and its consequences.
- [ADR-0002](./adr/0002-persistent-protected-rooms.md): optional SQLite persistence, protected sequential room IDs, and room lifecycle.
+1 -1
View File
@@ -28,7 +28,7 @@ Use separate control and media planes:
- Only pairs that cannot connect directly use authenticated TURN. Production requires STUN plus TURN over UDP and TCP; TURN/TLS is an optional restrictive-network enhancement, using its standard TCP port 5349 by default. Port 443 is optional and requires a dedicated public IP or a validated layer-4/SNI route when HTTPS already owns that address and port.
- Candidate selection is independent per pair. A room may simultaneously contain direct and relayed viewers without moving working peers onto the server.
- The deployed MVP broadcaster creates one peer connection per viewer. Rooms default to eight viewers and deployments may configure a limit from 1 through 16. Eight is an admission default, not a validated media-performance promise. A newer product target caps host media fanout at two; the current implementation does not satisfy that target above two viewers, and Proposed ADR-0004 owns the isolated experiment rather than silently changing this accepted baseline.
- An SFU is not the default path for the normal product envelope. Closed PR #12's explicit whole-room mode is superseded by merged PR #17's default-off automatic fallback. When a deployment configures SFU capacity, ADR-0005 permits selecting it only after cheaper endpoint-carried routes fail; that experiment is not deployed or accepted for production.
- An SFU is not the default path for the normal product envelope. Closed PR #12's explicit whole-room mode is superseded by merged PR #17's default-off automatic fallback. ADR-0005 permits selecting configured SFU capacity only after cheaper endpoint routes fail. The code ships in production but remains unconfigured, inactive, and unaccepted for production enablement.
## Consequences
@@ -51,14 +51,15 @@ LiveKit endpoint/key/secret tuple. There is no user-facing topology selector and
no process-wide `MEDIA_MODE` union. Without the tuple, the same controller ends
at peer assistance and bounded waiting/failure.
## Draft Implementation Status
## Implementation Status
Draft PR #17 (`feat/automatic-hybrid-routing`) implements this controller on top
of Draft PR #13. Draft PR #20 (`spike/hybrid-warm-fallback`) adds only the
bounded standby prewarm described below. Both pass CI but are not merged or
deployed. Production remains on the ordinary one-host-peer-per-viewer path.
Merged PR #17 implements this controller on top of merged PR #13. Merged PR #20
adds only the bounded standby prewarm described below. Their code ships in
production release `769de201f7cc`, but the deployment has neither
`PEER_ASSISTED_MEDIA` nor a LiveKit tuple, so its active path remains ordinary
one-host-peer-per-viewer P2P/TURN.
The Draft keeps the LiveKit dependency dormant unless the complete URL, API key,
The implementation keeps the LiveKit dependency dormant unless the complete URL, API key,
and API secret tuple is present together with `PEER_ASSISTED_MEDIA=true`. It
issues short-lived room-, role-, peer-, and publication-generation-bound grants,
and only allowlisted branch roots may subscribe. The server first retries a
@@ -199,7 +200,7 @@ second peer, then physically closed that new edge and reported its real
connection ID and active revision. The resulting plan had two allowlisted SFU
roots.
| Failure report to | Cold Draft PR #17 | Authenticated standby |
| Failure report to | Cold route controller | Authenticated standby |
| --- | ---: | ---: |
| SFU prepare | not retained | 5 ms |
| SFU active | 1,481 ms | 200 ms |
+3 -3
View File
@@ -24,7 +24,7 @@ Last updated: 2026-08-19
- Keep user-operated or central single-node SFU capacity optional. When configured, `main`'s #17 route controller may select it only as the final fallback; its fanout egress belongs to its operator and does not justify Redis or multi-node infrastructure. Closed PR #12's explicit whole-room SFU mode is superseded, not a current option.
- Keep automatic routing default-off behind ADR-0005: discrete failures drive peer recovery and optional SFU fallback, host media uses break-before-make under two active edges, and a binary relay capability keeps mobile/iPad clients as leaves. Merged PR #20's token-free prewarm passes the local sub-second gate; public transport and load gates remain.
- Prefer direct UDP, then TURN/UDP, with TURN/TCP as the required non-UDP fallback. Optional TURN/TLS uses TCP 5349 by default; TCP 443 needs a dedicated address or validated L4/SNI routing.
- Treat every quality setting as a ceiling. Production now defaults clarity-first with bounded manual controls and sender readback, but reported degradation remains unclassified. Run the 1/2/3-viewer plus TURN matrix before changing ceilings or adding an automatic controller.
- Treat video settings as ceilings. Production now defaults clarity-first with bounded manual controls and sender readback, but reported degradation remains unclassified. Keep browser audio request/presence-only: capture channels/rate are not portable controls, `maxBitrate` cannot raise quality, and Opus stereo/DTX/FEC lack portable setters. Run the 1/2/3-viewer plus TURN matrix before changing ceilings or adding an automatic controller.
- Do not add custom scene detection or dynamic-FPS control until WebRTC statistics and host resource measurements prove a material gap. Keep browser codec order until target hardware proves a more efficient common codec.
- Keep room policy deployment-driven. Public and password-only deployments use random temporary rooms. A site password plus SQLite path enables sequential persistent rooms; stopping sharing leaves the room and viewer link available.
@@ -69,6 +69,6 @@ These are measurement gates, not performance claims.
- Documentation index and current phase: `docs/README.md` and `docs/status.md`
- Requirements and design: `docs/需求理解.md` and `docs/方案设计.md`
- Media research: `docs/research/webrtc-p2p-screen-sharing.md`, `docs/research/realtime-quality-adaptation.md`, `docs/research/peer-assisted-media.md`, `docs/research/low-server-media-routes.md`, `docs/research/advanced-peer-distribution.md`, and `docs/research/native-shared-encode-sender.md`
- Architecture: ADR-0001, ADR-0002, proposed experiments ADR-0004/ADR-0005, and historical rejected/superseded ADR-0003
- Media research: `docs/research/webrtc-p2p-screen-sharing.md`, `docs/research/realtime-quality-adaptation.md`, `docs/research/browser-screen-audio-quality.md`, `docs/research/peer-assisted-media.md`, `docs/research/low-server-media-routes.md`, `docs/research/advanced-peer-distribution.md`, and `docs/research/native-shared-encode-sender.md`
- Architecture: ADR-0001/0002, proposed ADR-0004/0005, and rejected/superseded ADR-0003
- Deployment and maintenance: `docs/deployment.md`, `docs/maintenance.md`, `AGENTS.md`, and `.codex/`
@@ -0,0 +1,170 @@
# Browser Screen-Audio Quality Controls
Accessed: 2026-08-19
Status: retained no-go for user-facing browser audio quality controls.
## Scope And Decision
This review covers audio returned with `getDisplayMedia()` and sent through a
browser `RTCPeerConnection`. It does not decide whether the product ultimately
needs Windows per-application capture; that remains a separate native-sender
requirement.
The Web sender keeps the current minimal behavior:
- request audio with `audio: true`;
- treat an audio track as optional and warn before publishing when none exists;
- preserve returned audio through source changes, picture pause, P2P, relay, and
configured SFU routes; and
- expose no channel-count, sample-rate, Opus bitrate, stereo, DTX, or FEC
quality control.
No runtime or UI change is justified. In particular, Screener will not add
application-owned SDP munging for these controls or add a control whose value
cannot be separated into requested, applied, negotiated, and observed states
using standard APIs.
## Standard API Boundary
| Concern | Standard request or control | Honest readback | What it does not prove | Current product action |
| --- | --- | --- | --- | --- |
| Audio presence | `getDisplayMedia({ audio: true, video: ... })` expresses interest | `stream.getAudioTracks().length` proves only whether a track was returned | The browser may still return video only; a track does not identify system, window, tab, or selected-game audio | Keep the existing presence check and visible no-audio warning |
| Audio source choice | `systemAudio`, `windowAudio`, and `audioSelection` are picker hints that a user agent may ignore | There is no standard audio-source category readback corresponding to those hints | The selected audio scope, per-application isolation, or cross-browser availability | Do not present these hints as quality or source guarantees |
| Capture channels | The Screen Capture specification does not list generic `channelCount` as applicable to display audio | `track.getSettings().channelCount` may describe the returned track when the browser supplies it | That the app controlled the value, or that the RTP encoder sends stereo | Observe only in a future diagnostic; absent means unknown |
| Capture sample rate | The Screen Capture specification does not list generic `sampleRate` as applicable to display audio | `track.getSettings().sampleRate` may be present in an implementation | The Opus mode, RTP clock semantics, receiver output rate, or end-to-end fidelity | Observe only in a future diagnostic; absent means unknown |
| RTP send bitrate | `RTCRtpSender.setParameters()` can set `encodings[].maxBitrate` for audio | A following `getParameters()` can show the applied ceiling; outbound byte deltas show actual traffic | A target, minimum, quality increase, or Opus `maxaveragebitrate`; other limits may keep traffic lower | Do not add a control without a measured need to reduce audio bandwidth |
| Codec | The user agent chooses among negotiated send codecs unless a separately negotiated codec selection is available | `RTCCodecStats.mimeType` and `sdpFmtpLine` identify the codec and negotiated format parameters in use | That Opus was selected before stats exist, or that negotiated preferences describe actual content | Label only observed negotiated data; do not force a codec |
| Stereo | No stable sender parameter controls Opus stereo | Capture settings and negotiated codec/fmtp data can be inspected separately | Capture channel count does not prove encoded stereo; `opus/48000/2` does not prove stereo content | No control and no inferred stereo badge |
| DTX | The current WebRTC `RTCRtpEncodingParameters` dictionary has no `dtx` member | No portable standard sender readback proves DTX operation | A deprecated, non-standard browser field or SDK option is not a cross-browser contract | No control |
| FEC | There is no direct, portable audio FEC on/off sender parameter | Negotiated codec/fmtp data and inbound `fecPacketsReceived`/`fecBytesReceived` may provide evidence after use | Negotiation does not prove recovery occurred; zero counters do not prove FEC was disabled | No control; retain stats only as future diagnostic evidence |
The one technically portable write, audio `maxBitrate`, is not a quality-up
knob. The WebRTC specification defines it as a maximum and allows other limits
to constrain the sender further. It also warns that an audio ceiling below the
chosen encoding's needs may require playback to stop. Leaving it unset already
avoids an application-imposed ceiling, so adding a higher value cannot promise
better game audio.
## Capture And Browser Compatibility
The Screen Capture Working Draft says constraints are applied only after the
user chooses a surface and that `getUserMedia()` constraints do not apply to
display tracks unless this specification lists them. The only audio
constrainable properties it currently lists are `restrictOwnAudio` and
`suppressLocalAudioPlayback`; neither controls fidelity. It explicitly permits
a browser to return no audio despite an audio request.
MDN browser-compatibility data on the access date reports:
- display-audio capture in Chromium from Chrome 74, with whole-system audio on
Windows and ChromeOS but tab-only audio on Linux and macOS;
- no display-audio capture support in Firefox or Safari;
- `systemAudio` in desktop Chromium from Chrome 105, but not Firefox or Safari;
- `windowAudio` only partially implemented in Chrome 141: `"exclude"` and
`"system"` are supported, while `"window"` is not; and
- `MediaTrackSettings.channelCount` as non-Baseline because it is missing in
widely used browsers.
These are an implementation snapshot, not permanent product guarantees.
Chromium's own capture documentation also separates Windows/ChromeOS system
loopback capture from tab capture. It publishes no stable Web contract for a
fixed display-audio sample rate or channel count.
`encodings[].maxBitrate` has much broader compatibility: MDN data records
Chrome 69, Firefox 46, and Safari 11. The old `encodings[].dtx` field is absent
from Chrome and Firefox, present only as a deprecated non-standard Safari
feature, and absent from the current WebRTC IDL.
## Opus Signaling Is Not Source Truth
WebRTC endpoints must implement Opus, but negotiation may select another
mandatory audio codec. The application must wait for the in-use codec stats
before calling a stream Opus.
RFC 7587 requires an Opus SDP `a=rtpmap` clock rate of 48000 and channel count
of 2 for all Opus sessions. It separately explains that actual Opus media may
use another internal sampling rate. Therefore `audio/opus`, `clockRate=48000`,
and `channels=2` in codec stats do not establish a 48 kHz stereo capture or
stereo payload.
Opus `maxaveragebitrate`, `stereo`, `useinbandfec`, and `usedtx` are fmtp
offer/answer parameters. `stereo` is a receive preference, while
`sprop-stereo` is only a sender hint and explicitly not a guarantee. A browser
API that merely exposes the negotiated `sdpFmtpLine` can report those facts,
but does not provide a portable setter or prove the encoder's moment-to-moment
behavior.
## Reference Implementations
No code was copied. All repositories were inspected at pinned commits on
2026-08-19.
- LiveKit client-sdk-js, Apache-2.0, commit
[`0a2110d`](https://github.com/livekit/client-sdk-js/tree/0a2110d39904a06722a0c4d1ddbb9390bb06ad4d):
it treats
[`getSettings().channelCount` or requested constraints](https://github.com/livekit/client-sdk-js/blob/0a2110d39904a06722a0c4d1ddbb9390bb06ad4d/src/room/participant/LocalParticipant.ts#L899-L920)
as a stereo-input heuristic and populates standard
`RTCRtpEncodingParameters.maxBitrate`, but stereo/DTX/RED also travel in
LiveKit signaling. Its transport then
[munges Opus bitrate fmtp](https://github.com/livekit/client-sdk-js/blob/0a2110d39904a06722a0c4d1ddbb9390bb06ad4d/src/room/PCTransport.ts#L269-L333)
in SDP.
- lib-jitsi-meet, Apache-2.0, commit
[`63a04ec`](https://github.com/jitsi/lib-jitsi-meet/tree/63a04ecabd972ea75e877f9ba12086c13cb68210):
[`mungeOpus()`](https://github.com/jitsi/lib-jitsi-meet/blob/63a04ecabd972ea75e877f9ba12086c13cb68210/modules/RTC/TPCUtils.ts#L871-L929)
writes `stereo`, `sprop-stereo`, `maxaveragebitrate`, and `usedtx` into SDP,
including a Firefox-specific DTX exception.
- simple-peer, MIT, commit
[`f1a492d`](https://github.com/feross/simple-peer/tree/f1a492d1999ce727fa87193ebdea20ac89c1fc6d):
it has no higher-level audio quality contract and exposes only an advanced
generic
[`sdpTransform`](https://github.com/feross/simple-peer/blob/f1a492d1999ce727fa87193ebdea20ac89c1fc6d/index.js#L608-L664)
hook for SDP changes.
The Apache and MIT implementations support the same conclusion: a library can
offer product-specific audio modes, but the stereo/DTX/FEC/Opus-fmtp portion is
not equivalent to a portable browser sender control.
## Honest Future Diagnostics
If audio diagnostics become necessary, keep four layers separate:
1. **Capture:** track absent/present, plus `channelCount` and `sampleRate` only
when the browser returns numeric values.
2. **Requested/applied:** an RTP ceiling only if Screener actually calls
`setParameters()`, followed by immediate `getParameters()` readback.
3. **Negotiated:** the in-use codec MIME type and raw fmtp from `RTCCodecStats`,
labelled as negotiated parameters rather than actual stereo/DTX/FEC.
4. **Observed:** interval audio bitrate from byte deltas and receiver FEC
counters when implemented; missing values remain unknown.
Never substitute `48000`, `2`, `false`, or `0` for an unavailable field.
## Revisit Gates
Add a user-facing audio setting only when all of these are true:
1. A real Windows game/system-audio matrix identifies a reproducible user
problem and a desired outcome, such as a measured bandwidth reduction.
2. The setting uses a standards-track API without SDP munging or private
signaling and is supported across the declared sender-browser matrix.
3. The app can read back the applied value and separately observe the
negotiated codec and actual traffic or recovery behavior.
4. A controlled audible stereo/frequency fixture and packet-loss test shows
the intended result without silence, channel collapse, or route-specific
divergence.
## Primary Sources
- [W3C Screen Capture](https://www.w3.org/TR/screen-capture/)
- [W3C WebRTC](https://www.w3.org/TR/webrtc/)
- [W3C WebRTC Statistics](https://www.w3.org/TR/webrtc-stats/)
- [RFC 7587: RTP Payload Format for Opus](https://www.rfc-editor.org/rfc/rfc7587.html)
- [RFC 7874: WebRTC Audio Codec and Processing Requirements](https://www.rfc-editor.org/rfc/rfc7874.html)
- [MDN `getDisplayMedia()`](https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getDisplayMedia)
- [MDN `MediaTrackSettings.channelCount`](https://developer.mozilla.org/en-US/docs/Web/API/MediaTrackSettings/channelCount)
- [MDN Browser Compatibility Data: `MediaDevices`](https://github.com/mdn/browser-compat-data/blob/main/api/MediaDevices.json)
- [MDN Browser Compatibility Data: `RTCRtpSender`](https://github.com/mdn/browser-compat-data/blob/main/api/RTCRtpSender.json)
- [Chrome screen-sharing controls](https://developer.chrome.com/docs/web-platform/screen-sharing-controls)
- [Chrome 141 window-audio release note](https://developer.chrome.com/release-notes/141)
- [Chromium media-capture architecture index](https://chromium.googlesource.com/chromium/src/+/HEAD/docs/media/capture/)
+3 -2
View File
@@ -31,8 +31,9 @@ The smallest current plan is:
Any browser-spike failure other than isolated relay re-encoding closes that
browser-relay route. It does not cancel the separate native sender plan. Closed
PR #12's explicit whole-room SFU mode is superseded. ADR-0005 and merged PR #17
own the default-off automatic cross-mode controller; it remains undeployed.
Corrected localhost Chrome/LiveKit functional recovery passed, while public
own the default-off automatic cross-mode controller. Its code is deployed but
inactive because production has no peer-assist flag or LiveKit tuple. Corrected
localhost Chrome/LiveKit functional recovery passed, while public
transport, quality, load, and browser validation remain open.
## Token-Free SFU Standby Prewarm
+11 -6
View File
@@ -40,9 +40,9 @@ bitrate, and frame rate across all three profiles. This is a user observation,
not yet an instrumented result. The highest-confidence explanation is the
combination of one independent peer connection, sender, encoder pipeline, and
upload copy per viewer with `balanced` adaptation: host CPU or uplink pressure
can grow per edge, and the browser may then reduce both dimensions. The current
peer-assisted and SFU drafts are not deployed, so relay re-encoding and LiveKit
cannot explain that production observation.
can grow per edge, and the browser may then reduce both dimensions. Peer/SFU
code now ships in `769de201f7cc`, but its production configuration is absent;
relay re-encoding and LiveKit therefore cannot explain that earlier observation.
One controlled capture should classify the problem before changing constants:
@@ -107,9 +107,12 @@ resolution, frame rate, or bitrate.
- `maxBitrate` and `maxFramerate` are ceilings. They are neither minimums nor
target guarantees, and the project does not use SDP bitrate hacks.
- The folded advanced panel accepts only 720p/1080p/1440p, integer 15-60 fps,
2-12 Mbps, and the three preferences. Audio stereo/bitrate is not exposed
because standard sender controls cannot reliably guarantee the negotiated
browser audio mode.
2-12 Mbps, and the three preferences. It exposes no audio quality controls.
Display capture does not standardize channel-count or sample-rate control;
the portable audio `maxBitrate` field is only a ceiling and cannot raise
quality; and stereo, DTX, and FEC require negotiated fmtp/codec behavior that
has no portable sender setter. The full boundary is recorded in
`docs/research/browser-screen-audio-quality.md`.
- Every sender update derives from `getParameters()`, calls `setParameters()`,
then reads requested/applied bitrate, frame rate, scale, and preference.
Rejection or browser rewriting is visible rather than console-only.
@@ -141,6 +144,8 @@ CPU/GPU cost, public networks, or sustained behavior.
- No copied x264 CRF/preset recipe in the browser path.
- No forced codec order until target hardware measurements identify the actual
power-efficient encoder.
- No channel-count, sample-rate, Opus bitrate, stereo, DTX, or FEC control, and
no inference of actual stereo or sample rate from `opus/48000/2`.
- No promise that all browsers honor `applyConstraints` or degradation
preference identically.
- No automatic composite quality score or periodic profile controller before
+1 -1
View File
@@ -18,7 +18,7 @@ deployed binary still uses one host peer connection per viewer and P2P/TURN.
- Hidden routing is `direct P2P -> peer-assisted -> optional SFU`. After bounded ICE recovery it tries peer reparenting before an allowlisted SFU root. Session-bound revisions prepare, commit break-before-make under two host edges, or abort. Active SFU gets one token refresh, then fails back for that share.
- Complete fallback configuration adds a non-secret standby URL to peer-assisted authentication. Host and viewers import the SDK and make one token-free DNS/TLS warmup; no configuration means no field, import, request, participant, or media edge.
- A viewer starts as a leaf each session and explicitly advertises relay capacity zero or one; the Web client reports detected mobile/iPad clients as zero and desktop-class browsers as one. Withdrawal stops future assignment without moving a healthy edge. Browser relays remain one-child; the host remains two-child.
- Production uses one strict, memory-only quality setting with bounded manual ceilings and sender readback. It also carries equal idle-stage share/join actions and default-closed technical details.
- Production uses one strict, memory-only video quality setting with bounded manual ceilings and sender readback. It also carries equal idle-stage share/join actions and default-closed technical details. Browser audio remains request/presence-only.
## Verified Evidence
+4 -2
View File
@@ -139,11 +139,13 @@ Zod 在服务边界拒绝未知类型、超长字符串、非法 SDP/candidate
PoC 不强制 codec,也不做 SDP munging。先通过 stats 记录实际协商 codec、`encoderImplementation` 和 `powerEfficientEncoder`,再依据目标机器测量决定是否优先 H.264。这样避免静态 codec 偏好意外触发软件编码。
`audio: true` 只是请求。若 `getAudioTracks()` 为空,分享者界面必须显示“当前来源没有可共享音频”。纯 Web 首版接受浏览器提供的标签页或系统音频,不承诺仅捕获某个游戏进程。
`audio: true` 只是请求。若 `getAudioTracks()` 为空,分享者界面必须显示“当前来源没有可共享音频”。`systemAudio`、`windowAudio` 和 `audioSelection` 只影响选源器偏好,浏览器可以忽略且没有对应的标准音频来源类型读回,因此不能显示为系统、窗口或所选游戏音频保证。纯 Web 首版接受浏览器实际提供的标签页或系统音频,不承诺仅捕获某个游戏进程。
直播中切换来源仍由明确的按钮手势重新调用 `getDisplayMedia()`,不改变房间、host generation 或媒体拓扑。选源取消或失败时继续发送旧流;成功后为每个现有 sender 调用 `replaceTrack()`,健康连接不重新协商。初始 offer 预留 send-only audio transceiver,使音频可以在 `null -> track -> null` 间切换。若某个 peer 的替换或回滚失败,只重建该 peer;新流提交后才停止旧流,并以当前 stream 身份保护旧 track 的迟到 `ended` 事件。统计码率累加器在换源后重置,避免把不同 track 的计数器混算。
直播中切换质量不重新打开选源器:对当前视频 track 调用 `applyConstraints()`,再更新所有现有 sender 的参数;新加入、重建的 relay sender 和启用的 SFU publisher 使用同一权威设置。失败时保留当前分享并给出状态,不以重建整个房间补偿。临时暂停只禁用当前视频 track,使远端收到黑画面;音频、房间和连接保持不变,恢复时重新启用同一 track。浏览器标准 API 无法独立保证音频 stereo/bitrate 与最终协商 codec,因此当前高级区不暴露对应控件。
直播中切换质量不重新打开选源器:对当前视频 track 调用 `applyConstraints()`,再更新所有现有 sender 的参数;新加入、重建的 relay sender 和启用的 SFU publisher 使用同一权威设置。失败时保留当前分享并给出状态,不以重建整个房间补偿。临时暂停只禁用当前视频 track,使远端收到黑画面;音频、房间和连接保持不变,恢复时重新启用同一 track。
当前 Web 高级区不暴露音频质量控件。Screen Capture 规范未把 `channelCount`、`sampleRate` 列为 display-audio 可控属性;`track.getSettings()` 中偶尔出现的值只能作为捕获观测。标准 `RTCRtpSender.encodings[].maxBitrate` 可设置和读回,但只降低发送 ceiling,既不能要求编码器提高质量,也不等于 Opus `maxaveragebitrate`。Opus stereo、DTX 和 in-band FEC 属于协商的 codec/fmtp 行为,没有跨主流浏览器的标准 sender setter;`opus/48000/2` 也只是 RTP 信令约定,不能证明 48 kHz 源或实际 stereo。设计不为这些控件新增应用层 SDP munging,也不把 SDK 私有设置包装成浏览器保证。详细证据与复核门槛见 `docs/research/browser-screen-audio-quality.md`。
`frameRate` 约束和 sender 的 `maxFramerate` 都是目标或上限;浏览器仍控制实际捕获、重复帧处理、编码和拥塞降级。静态画面可能由浏览器减少实际编码数据,但标准没有保证每个实现以同一种方式节省采集或 GPU 开销。本 PoC 不添加画面差分、定时改约束或自定义动态 FPS 状态机,先用 `framesEncoded`、实际发送码率、`totalEncodeTime`、质量受限原因和主机资源占用测量真实问题。
+2 -1
View File
@@ -30,7 +30,7 @@
### P1 - 重要功能
- [ ] 质量控制扩展:推荐档位仍是默认入口;折叠高级区只允许 720p/1080p/1440p 分辨率 ceiling、15 至 60 的整数 FPS ceiling、2 至 12 Mbps 视频码率 ceiling,以及清晰/平衡/流畅三种偏好。直播中显式应用后才生效,避免滑杆产生连续控制流量。音频 stereo/bitrate 在标准浏览器 API 和实际协商 codec 不能可靠保证前只保留为后续测量项,不提供伪控制。实际质量持续低于所选上限时必须显示可解释的限制原因;在真实拥塞、编码器和接收端证据完成前,不把任一组合称为“完美平衡”。
- [ ] 质量控制扩展:推荐档位仍是默认入口;折叠高级区只允许 720p/1080p/1440p 分辨率 ceiling、15 至 60 的整数 FPS ceiling、2 至 12 Mbps 视频码率 ceiling,以及清晰/平衡/流畅三种偏好。直播中显式应用后才生效,避免滑杆产生连续控制流量。当前 Web 发送端不提供音频声道、采样率、Opus 码率、stereo、DTX 或 FEC 控件:display capture 未把声道和采样率定义为可移植控制,标准音频 `maxBitrate` 只是可能导致更低质量或停播的发送上限,其他项依赖 codec/fmtp 协商而没有可移植 sender setter。除非真实音频矩阵证明用户问题、标准 API 能应用并读回且实际结果可独立观测,否则不得为这些控件新增应用层 SDP munging 或用默认值制造伪控制。实际质量持续低于所选上限时必须显示可解释的限制原因;在真实拥塞、编码器和接收端证据完成前,不把任一组合称为“完美平衡”。
- [ ] Peer-assisted 媒体实验:分享端最多两个 child、每名 viewer 最多一个 child;信令服务只按深度和服务端加入序号确定性地形成两条 sticky 平衡链,不加入综合评分或主动重平衡。每个 viewer 将收到的标准 remote track 通过新的 `RTCPeerConnection` 转发,因此会逐跳解码和重编码;必须测量而不能称为 shared encode。服务端只在受房间数限制的内存状态中保存当前严格校验的质量设置,不写入 SQLite;host 鉴权后先重报设置,在线 viewers 接收更新,后加入者从鉴权快照取得当前值。该功能在 ADR-0004 的实验门槛通过并由后续 ADR 正式接受前不得成为产品路径。
- [ ] 原生 shared encode 分享端:后续独立的 Electron/原生分享端阶段必须保证一份兼容编码输出扇出到最多两套独立 WebRTC packetizer,并由单独 ADR 和测量约束。它降低分享端重复编码负载,但不消除每条出边各自的上传带宽。浏览器不保证跨 `RTCPeerConnection` 复用编码器,因此当前 Web spike 不实现、也不能宣称满足 shared encode 保证。
- [ ] 原生 shared encode relay:具备原生 relay 能力的 audience 同样最多维护两条下游媒体边,并应让一份兼容编码输出供两条边复用。普通浏览器 audience 继续作为 leaf 或使用当前会逐跳重编码的实验路径,不伪称 shared encode。
@@ -109,6 +109,7 @@
- Given 受控 RTT/丢包测试条件,When 连续分享高动态游戏画面 30 分钟,Then 延迟、帧率与丢帧达到上述目标,且没有不可恢复的冻结。
- Given 分享过程中切换 Wi-Fi/有线网络,When ICE 连接失败,Then 客户端触发 ICE restart,并在成功或超时后给出明确状态。
- Given 浏览器未返回系统音轨,When 分享开始,Then 分享者在启播前看到“当前来源没有可共享音频”的提示。
- Given 浏览器返回屏幕音轨,When 产品展示音频状态或未来连接详情,Then 只能把存在性、浏览器实际返回的 track settings、实际协商 codec/fmtp 和实际流量分别标为捕获、协商或观测事实;字段缺失必须显示为未知,且不得从 Opus 的 `48000/2` RTP 声明推断源采样率或实际 stereo。
### 仓库可移植性