feat(media): add explicit Web codec selection [skip ci] (#150)

This commit is contained in:
AAA authored and GitHub committed 2026-08-21 22:11:42 +08:00
1 parent f07871b73b
commit eef8d156f3
17 files changed
+233 -124

No files matched your search

+2 -2
View File
@@ -24,7 +24,7 @@ Last updated: 2026-08-21
- Use direct host P2P for one or two viewers, then the bounded controller for every room when `PEER_ASSISTED_MEDIA=true`. Room `1` is historical smoke, not a runtime gate; resource, quality, recovery, SFU/UDP, bounded-failure, and browser/mobile evidence remain separate.
- Browser relays resend remote `MediaStreamTrack` values and re-encode at each hop; WebRTC does not guarantee a shared encoder across peer connections, so measure the cost.
- Keep experiments bounded: the standard representation sequence is below; native RTP relay, encoded-object striping, and FEC remain separate.
- Production Web prefers H.264 through standard ordering, keeps the browser fallback list, and requests SFU H.264 with `backupCodec=false`; VP8 is used only when H.264 is unavailable. This is not a second encoder stack or shared-encode proof. ADR-0006's browser bridge remains no-go.
- Web defaults to browser/LiveKit codec negotiation; advanced settings offer Automatic, H.264, or VP8 for the next share while retaining fallback codecs and `backupCodec=false`. A codec choice neither proves hardware encoding nor shared encode. ADR-0006's browser bridge remains no-go.
- Mobile endpoints are Web Viewer-only. Current Android/iOS browsers do not expose Web Host capture; AirPlay/system mirroring is Viewer-local output.
- ADR-0005: direct/peer UDP -> bounded SFU roots -> selected-edge TURN. The deployed controller caps pending/answered `peer-selected` at one per room; Host ingress is independent. A pre-`ecc794d` exact-source/production-media canary proves active Host ingress only. Ordinary peers stay STUN-only.
- `PEER_ASSISTED_MEDIA=true` enables isolated all-room controllers; room `1` is historical. Keep sticky P2P and one recovery attempt per layer. Deployed healthy-SFU MBB covers one root and retained parent after reauth/recovery, gated by media proof, Host2/browser1 overlap and cooldown. It excludes `peer-selected`; active-SFU loss aborts into normal recovery. Capacity `0 -> 1`, multi-root and browser gates remain open.
@@ -51,7 +51,7 @@ Last updated: 2026-08-21
- Production requires independent `SITE_ACCESS_PASSWORD`: its stateless cookie authorizes creation/Host and code-only Viewer attempts; private grants stay direct, while public/private code entry needs site access alone/plus room password. Env/endpoint/cookie/nginx naming migrated atomically; failures remain neutral; there are no accounts/JWT/session rows. Local `gate:access-privacy` proves fragment isolation, site/grant transport-log absence, and raw room-password SQLite absence; room-password WebSocket/log ingress and live/headful evidence remain open.
- Production uses nginx, Node.js 24.19.0, coturn 4.17.2, and LiveKit 1.13.5 on UDP 7882 (TCP fallback off). Ordinary peer ICE receives no TURN; the selected-edge tuple is configured only for the current controller edge, with no credential pre-advertised to ordinary peers. Coturn retains the old authenticated-relay ports. Services are healthy with zero restarts.
- ADR-0005 rejects `PEER_ASSISTED_ROOM_IDS`; `PEER_ASSISTED_MEDIA=true` enables every room and ordinary peers stay STUN-only. Initial or active Host ingress may consume one bound relay-only grant; ready/abort clears it and failure restores peer baseline. Stale room-ID config fails startup; `screener-v2` is the only deployed wire.
- Defaults are clarity-first; balanced/fluid stay explicit. P2P writes before offer or explicit source/profile changes. Web SFU uses LiveKit 2.22.0 `q,h`; SFU/UDP and selected-TURN work; BWE is open. The lifecycle candidate retains sender/SDK/restart/republish state with rollback; Host A+B, capture and production remain open.
- Defaults are balanced with automatic codec; clarity/fluid and H.264/VP8 remain explicit. Codec is locked during a share. Web SFU uses LiveKit 2.22.0 `q,h`; SFU/UDP and selected-TURN work; BWE, Host A+B and production remain open.
- Chrome 151 five-Viewer 720p30 cap2 evidence: Host/relay fanout two; both relay children +80 decoded frames/~1.1 MB; every Viewer decoded; clean stop. Functional only; resource and heterogeneous-network behavior remain open.
## Provisional Quality Targets
+2 -2
View File
@@ -7,7 +7,7 @@ Last updated: 2026-08-21
Production at `https://share.bonfire.icu` is exact `6ccb516a47261054f91dfa2fafa408d39ced59fc`; immutable source/dist artifacts are 619,674/358,621 bytes, with hashes in deployment.
Cutover: 2,048 ms lock/1,272 ms stop-health/888 ms switch-health. SQLite v3/five rooms has SHA-256 `aef724ae52c4107be8ec8791e72f8dcaa11b0ec849fb432d022e2cfb9cfe0974`, owner/mode `screener:screener`/0600. Four services are active/running at `NRestarts=0`; rollbacks are `1331fbdd`, then `261e980` and `691863e`.
Ordinary ICE is STUN-only; limits are SFU roots <=2, one `peer-selected` attempt excluding Host ingress, and selected UDP TTL 120. This rollout ran no browser/media/SFU/TURN/performance canary; earlier exact-`16f6eab` Host-ingress proof remains separate.
Stable SFU, route/transport truth, preview notice, entry/nickname UI and Web relay cap2 are live. Native is source-only. Production prefers H.264 with negotiated VP8 fallback.
Stable SFU, route/transport truth, preview notice, entry/nickname UI and Web relay cap2 are live. Native is source-only. The controlled production A/B uses browser/LiveKit codec defaults; explicit codec controls are source-complete.
## Execution Principle
@@ -15,7 +15,7 @@ Parallel flagship work; minimum relevant checks and rollback proof. Research plu
## Current Snapshot
- Capture precedes rooms; source/quality changes and AV pause preserve peers. Defaults are clarity-first; balanced/fluid stay explicit. P2P writes before offer or explicit changes, never on answer. The SFU lifecycle candidate retains sender/SDK/restart/republish options with rollback. Browser degradation and Host SFU A+B remain open.
- Capture precedes rooms; source/quality changes and AV pause preserve peers. Defaults are balanced and automatic codec; clarity/fluid plus H.264/VP8 stay explicit, with codec locked during a share. Browser degradation and Host SFU A+B remain open.
- Production `screener-v2` admits private fragment grants directly. Code-only Viewers need site access, then public rooms accept the code while private rooms also require their password. `SITE_ACCESS_PASSWORD`, `/api/site-access`, and its cookie are deployed atomically; raw credentials are never stored.
- Source UI uses adjacent nickname editing, distinct site/room prompts, and a shared room-code form. Status and details stay neutral before media proof; current evidence labels green P2P/yellow `SFU fallback`, while relay evidence adds `TURN` plus only a locally observed protocol.
- Native v2 is source-only (memory rooms, 300s reclaim); it follows authenticated direct-child assignments up to two and fails unsupported SFU/selected ingress boundedly.
+5 -5
View File
@@ -104,7 +104,7 @@ Web Host 在 `authenticate` 中显式发送 `viewerPresence: true` capability,
## 信令协议
WebSocket 使用无 query/fragment 的同源 `/signal`。upgrade 只记录 site-access cookie 是否有效;连接后五秒内第一条消息必须是 `screener-v2` 的房间级 `authenticate`。Host 消息携带 room、role、内部 Host token 与 clientId,并由 socket state 复核 site access;Web Host 可选发 `viewerPasswordSettings: true` capability,Native v2 不发送且不接收任何密码设置新消息。Viewer 消息携带 room、role、clientId,以及 private room 的 Viewer grant 或当前尝试的房间密码。服务端设置 64 KiB `maxPayload`、关闭 `perMessageDeflate`,并以 ping/pong 清理失效连接。单进程固定限制 2048 条总连接和 256 条未鉴权连接,容量在 upgrade 前预留,超限直接返回 HTTP 503 并销毁 socket。
WebSocket 使用无 query/fragment 的同源 `/signal`。upgrade 只记录 site-access cookie 是否有效;连接后五秒内第一条消息必须是 `screener-v3` 的房间级 `authenticate`。Host 消息携带 room、role、内部 Host token 与 clientId,并由 socket state 复核 site access;Web Host 可选发 `viewerPasswordSettings: true` capability,Native v2 不发送且不接收任何密码设置新消息。Viewer 消息携带 room、role、clientId,以及 private room 的 Viewer grant 或当前尝试的房间密码。服务端设置 64 KiB `maxPayload`、关闭 `perMessageDeflate`,并以 ping/pong 清理失效连接。单进程固定限制 2048 条总连接和 256 条未鉴权连接,容量在 upgrade 前预留,超限直接返回 HTTP 503 并销毁 socket。
主要消息:
@@ -152,11 +152,11 @@ Zod 在服务边界拒绝未知类型、超长字符串、非法 SDP/candidate
| 1080p30 | 1920x1080、30 fps | 5 Mbps/peer |
| 720p30 | 1280x720、30 fps | 3 Mbps/peer |
浏览器可返回更低设置,界面以 `track.getSettings()` 显示实际捕获结果。Current source 保留有界手动质量控制:视频 track 使用 `contentHint = "motion"`,返回的音频 track 使用 `contentHint = "music"`;二者都是内容提示而不是 codec 或质量档位。三档推荐质量与折叠高级区初始值统一采用 `degradationPreference = "maintain-resolution"`(clarity-first),并允许显式选择 `balanced` 或 `maintain-framerate`(fluid)。高级区将设置严格限制为 720p/1080p/1440p、15 至 60 整数 FPS、2 至 12 Mbps,以及 `maintain-resolution`、`balanced`、`maintain-framerate` 三种偏好。它们只是 ceiling 和浏览器偏好,不保证实际分辨率、帧率或码率。本期屏幕音频只有一个固定媒体默认,不改面板名称也不增加音频 UI。
浏览器可返回更低设置,界面以 `track.getSettings()` 显示实际捕获结果。Current source 保留有界手动质量控制:视频 track 使用 `contentHint = "motion"`,返回的音频 track 使用 `contentHint = "music"`;二者都是内容提示而不是 codec 或质量档位。三档推荐质量与折叠高级区初始值统一采用 `degradationPreference = "balanced"`,并允许显式选择 clarity-first 或 fluid。高级区将设置限制为 720p/1080p/1440p、15 至 60 整数 FPS、2 至 12 Mbps、三种 degradation preference,以及 `自动 | H.264 | VP8`;codec 直播中锁定并于下一次分享生效。它们只是 ceiling 和浏览器偏好,不保证实际分辨率、帧率、码率或硬编。本期屏幕音频只有一个固定媒体默认,不增加音频 UI。
每次 `setParameters()` 都必须从当次 `getParameters()` 派生更新,然后读回 requested/applied `maxBitrate`、`maxFramerate`、`scaleResolutionDownBy` 和 degradation preference。浏览器负责具体降级,实际结果由 stats 观测;拒绝或浏览器改写必须进入可见状态,不能只写 `console.warn`。
Web 视频端不做 SDP munging。P2P 与浏览器 relay 在首个 offer 前读取标准 sender capabilities;存在 H.264 时用 `setCodecPreferences()` 将全部 H.264 条目置前,同时保留完整的 browser codec/repair 列表,能力缺失或 API 拒绝时维持浏览器默认。SFU publication 同样在 H.264 能力存在时请求 `videoCodec: "h264"`,否则请求 VP8,并始终保持 `backupCodec: false`,因此不会为兼容回退同时维护第二份 codec publication。实际 outbound codec/profile 与 stats 是协商结果真相。这个可逆偏好只要求定向互通和回退验证;硬件、CPU/GPU、游戏 FPS 和目标网络证据用于性能结论及未来动态选择,不作为上线前置。AV1 仍不得仅凭压缩率推断默认启用。唯一获准的 Web SDP 变换是下述 Viewer answer 的 Opus stereo receive preference,不得借此扩展视频码率、codec 或通用 munging。
Web 视频端不做视频 SDP munging。`automatic` 不调用 P2P `setCodecPreferences()` 且不覆盖 LiveKit 默认 codec;显式 H.264/VP8 在首个 offer/publication 前应用,P2P 与 browser relay 仍保留完整 browser codec/repair 回退列表,SFU 始终保持 `backupCodec: false`。codec 改动不热切换现有连接。实际 outbound codec/profile、encoder implementation 与 stats 是结果真相;浏览器支持 H.264 不证明命中硬编。AV1 仍不得仅凭压缩率推断默认启用。唯一获准的 Web SDP 变换是下述 Viewer answer 的 Opus stereo receive preference,不得借此扩展视频码率、codec 或通用 munging。
`audio: true` 只是请求。若 `getAudioTracks()` 为空,分享者界面必须显示“当前来源没有可共享音频”。Web 端同时传入 `windowAudio: "window"` 与 `systemAudio: "include"`:窗口选项请求该窗口/应用音频,整个屏幕选项允许用户勾选系统音频;标签页使用浏览器自身的标签页音频选择。浏览器可以忽略这些 picker hints,且没有对应的标准来源类型读回,因此 UI 不额外推断或标注来源。系统音频由用户显式选择,可能包含语音或通知;video-only 始终可用。
@@ -343,7 +343,7 @@ hop 按[低服务器成本媒体路由](./research/low-server-media-routes.md)
- RoomStore 的随机临时房间、TTL、Host token/Viewer grant 摘要验证、private/public policy、grant 到期/跨房拒绝、rotate/revoke 强撤销、停止分享等待、Viewer 上限和超额拒绝;
- SQLite 房间库从 `1` 自增、重启恢复、v1/v2 -> v3 单事务迁移、nullable 32-byte grant 与 48-byte password material CHECK、畸形值 fail-closed、rotate/revoke/password 写失败原子性和数据最小化;数据库路径无 site access 时启动失败;
- STUN/TURN UDP URL 严格校验、生产 STUN 必需、selected-edge tuple 完整性/TTL/secret 独立性、issuer HMAC/expiry/opaque tag、当前 route/session/revision/old-new connection generation 与 one-use rebuild guard,以及 ordinary peer、未命中房间和 Native 的 STUN-only isolation;LiveKit `sfu-config` 保持独立边界;
- 单一 `screener-v2` 信令 literal、v1 旧页面 mismatch 的一次性刷新终态,以及协议非法 JSON、未知消息、错误方向、跨房间目标和 payload 限制;
- 单一 `screener-v3` 信令 literal、v2 旧页面 mismatch 的一次性刷新终态,以及协议非法 JSON、未知消息、错误方向、跨房间目标和 payload 限制;
- 质量协议的 host 授权、严格字段与数值范围、last-wins、当前/未来 relay、已配置 SFU 和普通 P2P wire 兼容;sender 的四字段 requested/applied 读回、拒绝与持续受限提示;screen audio SDP helper 覆盖 CRLF、existing/missing fmtp、unknown key preservation、幂等、payload number、rejected/non-audio,以及 empty/no-Opus/ambiguous/parser 异常原对象 fallback,`ViewerPeer` 覆盖 initial/restart/rebuild 时同一结果的 local/signaling 一致性,SFU publisher 精确断言 high-quality stereo/forceStereo/DTX 且不覆盖 RED;
- HTTP/WS 集成:nginx per-source site-access `limit_req` 与统一错误不泄露 secret,应用不建立 IP limiter 状态;Viewer 可在 Origin/容量门下 upgrade;Host 首条鉴权同时要求 upgrade-captured site access 与 Host token,Viewer 要么持 exact-room grant,要么先有 site access 再按 room policy;site-access cookie 不绕过 private grant/password,grant/password 不能建房、取得 Host 或跨房。覆盖 fragment/sessionStorage/replaceState、无 site access 的 public/password code-only 统一拒绝、通过 site access 后的 public/password 进入、不含明文于 URL/浏览器持久存储/SQLite/log、v2 Native wire,以及 rotate/revoke 后 grace/route 清理、旧 Viewer/上游媒体边关闭和 reconnect 拒绝;
- 客户端策略与 stats 解析:会话替换不重连,ordinary ICE、LiveKit 与 selected-edge TURN protocol 分离,远端 TURN transport 保持未知;
@@ -400,7 +400,7 @@ STUN-only;只有 controller-selected edge 可接收一次短期 grant。该边
固定 `2` 是当前上线档而非永久产品上限;后续统一 evidence-driven controller 可在独立验收后使用离散 `0/1/2/3`,不按 device/UA 分支,并由持续 sender bitrate、`availableOutgoingBitrate`、区间 encode time、`qualityLimitationReason`、loss/RTT 健康窗、server deployment hard ceiling 与 cooldown 共同限界。本次 slice 不实现该动态控制。H.264、hardware encode 或 shared encode 可降低编码成本或同观感码率,但每个 child 的网络副本仍独立存在。
Peer-assisted 信令只接受一个严格对象:`resolution` 为 `720p | 1080p | 1440p`,`maxFramerate` 为 15 至 60 的整数,`maxBitrate` 为 2,000,000 至 12,000,000 的整数,`degradationPreference` 为 `maintain-resolution | balanced | maintain-framerate`,缺字段、多余字段和越界值都拒绝。服务端用受房间容量限制的内存 Map 保存完整设置,默认 1080p60、8 Mbps、`maintain-resolution`,不写入 SQLite;停止分享时保留,房间被放弃或过期时删除。只有 peer-assisted host 可更新,鉴权快照和在线广播携带同一完整设置;普通 P2P 的严格鉴权 wire 保持原样,host 或 viewer 在该模式发送同步消息都会被拒绝。启用 SFU 时 publisher 也使用这一设置,不另建一套质量协议。
Peer-assisted 信令只接受一个严格对象:`resolution` 为 `720p | 1080p | 1440p`,`maxFramerate` 为 15 至 60 的整数,`maxBitrate` 为 2,000,000 至 12,000,000 的整数,`degradationPreference` 为 `maintain-resolution | balanced | maintain-framerate`,可选 `videoCodec` 为 `automatic | h264 | vp8`,其余字段和越界值拒绝。服务端用房间内存 Map 保存完整设置,默认 1080p60、8 Mbps、`balanced`、`automatic`,不写 SQLite;停止分享时保留,房间被放弃或过期时删除。只有 peer-assisted host 可更新,鉴权快照和在线广播携带同一设置;本 wire 由 `screener-v3` 隔离旧页面,普通 P2P 严格鉴权结构不变。SFU publisher 复用同一设置。
Viewer 在处理 peer-assisted 鉴权快照时先更新 `ViewerRelay` 的 desired settings,再应用拓扑分配;同一个串行队列协调 remote stream、当前 child 更新和以后重建的 child,因此它们读取同一最新目标。每个 `HostPeer` 也把初始 sender 配置、`replaceStream()` 和 `updateProfile()` 放进同一 sender-mutation queue;desired settings 在入队前同步更新,已排队操作执行时读取最新值,使快速连续修改保持 last-wins,而不增加版本号或确认协议。这里同步的是浏览器 sender 的目标上限,不是实际分辨率、帧率、码率或 shared encoding 的性能保证。
+1 -1
View File
@@ -55,7 +55,7 @@
- [ ] Windows 应用音频:Windows 11 Native sender 使用本地 opaque window target 绑定 HWND、PID 与进程创建时间,只捕获所选游戏及其子进程的音频,避免把语音聊天或通知声一并分享;“全部系统声音”只能由分享者显式选择。普通 Windows 10 不具备当前 process-loopback API 门槛,必须明确标为未解决/不支持而不是伪兼容。浏览器无法证明或隔离任意进程音频时必须如实提示,原生捕获失败、无声或进程退出不得静默扩大为整机捕获。
- [ ] 连接自检:在开始分享前验证信令、STUN 和 SFU ICE/UDP 可达性,并给出不暴露地址或凭据的诊断结果。启用 selected-edge TURN canary 时只报告脱敏的授权阶段与 selected candidate type,不输出 URL、username、credential、地址或原始 candidate;自检只覆盖当前 UDP 媒体梯级。
- [ ] 受限网络增强:短期鉴权 TURN 只能由完整、默认关闭的部署 tuple 与当前 room controller 启用,并且只能在 direct/peer UDP 与 SFU/UDP 均已按有界梯级失败后,向 controller 选中的当前异常 edge 一次性授权。普通 peer PC 从创建起保持 STUN-only;不得向整个 participant、房间或所有 peer connections 预发 TURN candidates。凭据必须绑定当前授权 session 与 route/connection generation、短期有效、内存态、一次性消费且不进入 URL、日志、浏览器持久化或 SQLite。媒体保持 UDP-only,全部路径耗尽后明确失败;TURN 不得被描述为更高清的传输方式。
- [x] Web H.264 优先与协商回退:P2P/浏览器 relay 在首个 offer 前用标准 `setCodecPreferences()` 把浏览器发送能力中的全部 H.264 条目置前,同时保留完整的 repair 与其他 codec 列表;SFU 在端点支持 H.264 时请求 H.264 并保持 `backupCodec = false`。能力缺失或服务端不能采用时回退 VP8,不维护第二套应用编码链。实际协商 codec/profile 和 stats 是结果真相;目标设备/生产证据约束性能宣传与未来自适应选择,但完整 CPU/GPU/游戏矩阵不再阻塞这个可逆标准偏好。AV1 仍不得仅凭压缩率推断默认启用。
- [x] Web 编码选择:高级视频设置提供 `自动 | H.264 | VP8`,默认自动使用浏览器/LiveKit 协商;显式选择只在下一次分享前用标准 codec preference 生效,直播中锁定。P2P/浏览器 relay 保留完整 repair 与其他 codec 回退列表,SFU 保持 `backupCodec = false`,不维护第二套应用编码链。实际 codec/profile、encoder implementation 与 stats 是结果真相;H.264 可选不等于浏览器必走硬编,AV1 仍不得仅凭压缩率推断默认启用。
- [ ] 诊断页面:用户可导出不含敏感地址和凭据的连接诊断信息。
### P2 - 可选功能
+21 -4
View File
@@ -4,6 +4,7 @@ import {
type QualityProfileId,
type QualityResolution,
type QualitySettings,
type VideoCodecPreference,
} from "../../shared/protocol";
import { displayMediaOptions } from "./audio-capture";
@@ -12,6 +13,7 @@ export type {
QualityProfileId,
QualityResolution,
QualitySettings,
VideoCodecPreference,
} from "../../shared/protocol";
export type QualityProfile = QualitySettings;
@@ -22,13 +24,15 @@ export const QUALITY_PROFILES = {
resolution: "1080p",
maxFramerate: 30,
maxBitrate: 5_000_000,
degradationPreference: "maintain-resolution",
degradationPreference: "balanced",
videoCodec: "automatic",
},
"720p30": {
resolution: "720p",
maxFramerate: 30,
maxBitrate: 3_000_000,
degradationPreference: "maintain-resolution",
degradationPreference: "balanced",
videoCodec: "automatic",
},
} as const satisfies Record<QualityProfileId, QualitySettings>;
@@ -53,6 +57,12 @@ export const DEGRADATION_PREFERENCE_LABELS = {
"maintain-framerate": "流畅优先",
} as const satisfies Record<DegradationPreference, string>;
export const VIDEO_CODEC_PREFERENCE_LABELS = {
automatic: "自动",
h264: "H.264",
vp8: "VP8",
} as const satisfies Record<VideoCodecPreference, string>;
export interface VideoSenderParameterValues {
maxBitrate: number | null;
maxFramerate: number | null;
@@ -87,7 +97,8 @@ export function qualitySettingsEqual(
left.resolution === right.resolution &&
left.maxFramerate === right.maxFramerate &&
left.maxBitrate === right.maxBitrate &&
left.degradationPreference === right.degradationPreference
left.degradationPreference === right.degradationPreference &&
(left.videoCodec ?? "automatic") === (right.videoCodec ?? "automatic")
);
}
@@ -95,7 +106,13 @@ export function matchingQualityProfileId(
settings: QualitySettings,
): QualityProfileId | null {
for (const id of Object.keys(QUALITY_PROFILES) as QualityProfileId[]) {
if (qualitySettingsEqual(settings, QUALITY_PROFILES[id])) {
const profile = QUALITY_PROFILES[id];
if (
settings.resolution === profile.resolution &&
settings.maxFramerate === profile.maxFramerate &&
settings.maxBitrate === profile.maxBitrate &&
settings.degradationPreference === profile.degradationPreference
) {
return id;
}
}
+59 -1
View File
@@ -80,6 +80,8 @@ import {
type DegradationPreference,
type QualityProfileId,
type QualitySettings,
type VideoCodecPreference,
VIDEO_CODEC_PREFERENCE_LABELS,
} from "../media/quality";
import { HostSfuRoute } from "../media/host-sfu-route";
import {
@@ -607,7 +609,24 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) {
setAdvancedQuality(settings);
}
function changeVideoCodec(videoCodec: VideoCodecPreference): void {
if (phase === "starting" || phase === "live") {
return;
}
const next = { ...qualitySettingsRef.current, videoCodec };
qualitySettingsRef.current = next;
setQualitySettings(next);
setAdvancedQuality((current) => ({ ...current, videoCodec }));
}
async function changeQuality(nextProfile: QualitySettings): Promise<void> {
if (
phase === "live" &&
(nextProfile.videoCodec ?? "automatic") !==
(qualitySettingsRef.current.videoCodec ?? "automatic")
) {
return;
}
if (phase !== "live") {
commitQuality(nextProfile);
return;
@@ -1807,7 +1826,13 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) {
switchingSource ||
changingQuality
}
onClick={() => void changeQuality(QUALITY_PROFILES[id])}
onClick={() =>
void changeQuality({
...QUALITY_PROFILES[id],
videoCodec:
qualitySettingsRef.current.videoCodec ?? "automatic",
})
}
>
{QUALITY_PROFILE_LABELS[id]}
</button>
@@ -1915,6 +1940,39 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) {
))}
</div>
</fieldset>
<fieldset className="control-group quality-priority">
<legend>视频编码</legend>
<div className="segmented-control">
{(
Object.keys(
VIDEO_CODEC_PREFERENCE_LABELS,
) as VideoCodecPreference[]
).map((codec) => (
<button
key={codec}
type="button"
className={
(advancedQuality.videoCodec ?? "automatic") === codec
? "is-selected"
: undefined
}
aria-pressed={
(advancedQuality.videoCodec ?? "automatic") === codec
}
disabled={
phase === "starting" ||
phase === "live" ||
changingQuality
}
onClick={() =>
changeVideoCodec(codec)
}
>
{VIDEO_CODEC_PREFERENCE_LABELS[codec]}
</button>
))}
</div>
</fieldset>
<button
className="button button-secondary"
type="button"
+3 -2
View File
@@ -14,7 +14,6 @@ import {
type QualityProfile,
type VideoSenderParameterReadback,
} from "../media/quality";
import { preferredVideoCodec } from "../webrtc/video-codec-preference";
export interface SfuConnectionConfig {
url: string;
@@ -611,7 +610,9 @@ function videoPublishOptions(
const highResolution = QUALITY_RESOLUTIONS[profile.resolution];
return {
backupCodec: false,
videoCodec: preferredVideoCodec(),
...(!profile.videoCodec || profile.videoCodec === "automatic"
? {}
: { videoCodec: profile.videoCodec }),
simulcast: true,
screenShareEncoding: {
maxBitrate: profile.maxBitrate,
+5 -2
View File
@@ -15,7 +15,7 @@ import {
collectConnectionMetrics,
createStatsAccumulator,
} from "./stats";
import { preferH264 } from "./video-codec-preference";
import { applyVideoCodecPreference } from "./video-codec-preference";
const MAX_PENDING_CANDIDATES = 64;
type PeerIceConfig = Pick<RTCConfiguration, "iceServers">;
@@ -118,7 +118,10 @@ export class HostPeer {
direction: "sendonly",
streams: [this.stream],
});
preferH264(videoTransceiver);
applyVideoCodecPreference(
videoTransceiver,
this.desiredProfile.videoCodec,
);
this.videoSender = videoTransceiver.sender;
this.audioSender = this.connection.addTransceiver(audioTrack ?? "audio", {
direction: "sendonly",
+18 -16
View File
@@ -1,4 +1,4 @@
export type PreferredVideoCodec = "h264" | "vp8";
import type { VideoCodecPreference } from "../../shared/protocol";
function videoCodecs(): RTCRtpCodec[] {
if (typeof RTCRtpSender === "undefined") {
@@ -7,27 +7,29 @@ function videoCodecs(): RTCRtpCodec[] {
return RTCRtpSender.getCapabilities?.("video")?.codecs ?? [];
}
function isH264(codec: RTCRtpCodec): boolean {
return codec.mimeType.toLowerCase() === "video/h264";
}
export function preferredVideoCodec(): PreferredVideoCodec {
return videoCodecs().some(isH264) ? "h264" : "vp8";
}
export function preferH264(transceiver: RTCRtpTransceiver): boolean {
if (typeof transceiver.setCodecPreferences !== "function") {
return false;
export function applyVideoCodecPreference(
transceiver: RTCRtpTransceiver,
preference: VideoCodecPreference | undefined,
): boolean {
const resolvedPreference = preference ?? "automatic";
if (
resolvedPreference === "automatic" ||
typeof transceiver.setCodecPreferences !== "function"
) {
return resolvedPreference === "automatic";
}
const codecs = videoCodecs();
const h264 = codecs.filter(isH264);
if (h264.length === 0) {
const mimeType = `video/${resolvedPreference}`;
const preferred = codecs.filter(
(codec) => codec.mimeType.toLowerCase() === mimeType,
);
if (preferred.length === 0) {
return false;
}
try {
transceiver.setCodecPreferences([
...h264,
...codecs.filter((codec) => !isH264(codec)),
...preferred,
...codecs.filter((codec) => codec.mimeType.toLowerCase() !== mimeType),
]);
return true;
} catch {
+1 -10
View File
@@ -279,16 +279,7 @@ export class SignalingServer {
try {
message = decodeClientMessage(encoded);
} catch {
if (!state.authenticated) {
this.sendError(
socket,
"AUTH_REQUIRED",
"页面版本已更新,请刷新后重试",
);
socket.close(4001, "Protocol mismatch");
} else {
this.rejectInvalidMessage(socket);
}
this.rejectInvalidMessage(socket);
return;
}
if (
+13 -2
View File
@@ -5,7 +5,7 @@ import { isCanonicalVideoCodecEvidence } from "./video-codec-evidence.js";
export const MAX_VIEWERS_PER_ROOM_LIMIT = 16;
export const MAX_PARTICIPANTS_PER_ROOM_LIMIT = MAX_VIEWERS_PER_ROOM_LIMIT + 1;
export const MAX_SIGNAL_BYTES = 64 * 1024;
export const SIGNALING_PROTOCOL = "screener-v2";
export const SIGNALING_PROTOCOL = "screener-v3";
export const ROOM_CODE_LENGTH = 12;
export const MAX_MEDIA_ROUTE_REVISION = Number.MAX_SAFE_INTEGER;
export const MAX_SFU_TOKEN_LENGTH = 8 * 1024;
@@ -166,12 +166,22 @@ export type DegradationPreference = z.infer<
typeof degradationPreferenceSchema
>;
export const videoCodecPreferenceSchema = z.enum([
"automatic",
"h264",
"vp8",
]);
export type VideoCodecPreference = z.infer<
typeof videoCodecPreferenceSchema
>;
export const qualitySettingsSchema = z
.object({
resolution: qualityResolutionSchema,
maxFramerate: z.number().int().min(15).max(60),
maxBitrate: z.number().int().min(2_000_000).max(12_000_000),
degradationPreference: degradationPreferenceSchema,
videoCodec: videoCodecPreferenceSchema.optional(),
})
.strict();
export type QualitySettings = z.infer<typeof qualitySettingsSchema>;
@@ -179,7 +189,8 @@ export const DEFAULT_QUALITY_SETTINGS = {
resolution: "1080p",
maxFramerate: 60,
maxBitrate: 8_000_000,
degradationPreference: "maintain-resolution",
degradationPreference: "balanced",
videoCodec: "automatic",
} as const satisfies QualitySettings;
export const relayDownstreamEdgesSchema = z.union([
+1 -1
View File
@@ -541,7 +541,7 @@ describe("client signaling recovery policy", () => {
JSON.parse(String(sockets[0]!.send.mock.calls[0]![0])),
).toMatchObject({
type: "authenticate",
protocol: "screener-v2",
protocol: "screener-v3",
});
const message = new Event("message");
Object.defineProperty(message, "data", { value: payload });
+49 -36
View File
@@ -1,6 +1,9 @@
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { QUALITY_PROFILES } from "../src/client/media/quality.ts";
import {
QUALITY_PROFILES,
type QualityProfile,
} from "../src/client/media/quality.ts";
import type { PeerSnapshot } from "../src/client/types.ts";
import { HostPeer } from "../src/client/webrtc/host-peer.ts";
import { ViewerRelay } from "../src/client/webrtc/viewer-relay.ts";
@@ -116,9 +119,9 @@ class FakePeerConnection {
this.transceiverInputs.push({ trackOrKind, init });
return {
sender,
setCodecPreferences: vi.fn((codecs: RTCRtpCodec[]) => {
setCodecPreferences: (codecs: RTCRtpCodec[]) => {
this.codecPreferenceCalls.push([...codecs]);
}),
},
} as unknown as RTCRtpTransceiver;
}
@@ -324,12 +327,13 @@ function createPeer(
stream: MediaStream,
onUpdate: (snapshot: PeerSnapshot) => void = () => undefined,
iceConfig: IceConfig = { iceServers: [] },
profile: QualityProfile = QUALITY_PROFILES["720p30"],
): HostPeer {
return new HostPeer(
"viewer-peer",
iceConfig,
stream,
QUALITY_PROFILES["720p30"],
profile,
{
sendSignal: () => true,
onUpdate,
@@ -346,19 +350,13 @@ beforeEach(() => {
statsCallbacks.length = 0;
vi.stubGlobal("RTCPeerConnection", FakePeerConnection);
vi.stubGlobal("RTCRtpSender", {
getCapabilities: vi.fn(() => ({
getCapabilities: () => ({
codecs: [
{ mimeType: "video/VP8", clockRate: 90_000 },
{ mimeType: "video/rtx", clockRate: 90_000, sdpFmtpLine: "apt=96" },
{
mimeType: "video/H264",
clockRate: 90_000,
sdpFmtpLine: "packetization-mode=1;profile-level-id=42e01f",
},
{ mimeType: "video/rtx", clockRate: 90_000, sdpFmtpLine: "apt=102" },
{ mimeType: "video/H264", clockRate: 90_000 },
],
headerExtensions: [],
})),
}),
});
vi.stubGlobal("window", {
setInterval: vi.fn((callback: () => void) => {
@@ -378,27 +376,7 @@ afterEach(() => {
});
describe("HostPeer source replacement", () => {
it("prefers H.264 before the first offer while retaining codec fallback", async () => {
const peer = createPeer(createStream(createTrack("video", "video"), null));
await expect(peer.start()).resolves.toBe(true);
const preferences = FakePeerConnection.latest!.codecPreferenceCalls[0]!;
expect(preferences).toHaveLength(4);
expect(preferences[0]?.mimeType).toBe("video/H264");
expect(preferences.map(({ mimeType }) => mimeType)).toContain("video/VP8");
expect(
preferences.filter(({ mimeType }) => mimeType === "video/rtx"),
).toHaveLength(2);
});
it("keeps browser defaults when H.264 send capability is absent", async () => {
vi.stubGlobal("RTCRtpSender", {
getCapabilities: vi.fn(() => ({
codecs: [{ mimeType: "video/VP8", clockRate: 90_000 }],
headerExtensions: [],
})),
});
it("leaves browser codec ordering unchanged", async () => {
const peer = createPeer(createStream(createTrack("video", "video"), null));
await expect(peer.start()).resolves.toBe(true);
@@ -406,6 +384,41 @@ describe("HostPeer source replacement", () => {
expect(FakePeerConnection.latest!.codecPreferenceCalls).toEqual([]);
});
it.each(["h264", "vp8"] as const)(
"prefers %s before the first offer while retaining fallback codecs",
async (videoCodec) => {
vi.stubGlobal("RTCRtpSender", {
getCapabilities: () => ({
codecs: [
{ mimeType: "video/VP8", clockRate: 90_000 },
{ mimeType: "video/rtx", clockRate: 90_000 },
{
mimeType: "video/H264",
clockRate: 90_000,
sdpFmtpLine: "packetization-mode=1;profile-level-id=42001f",
},
{ mimeType: "video/rtx", clockRate: 90_000 },
],
headerExtensions: [],
}),
});
const peer = createPeer(
createStream(createTrack("video", "video"), null),
() => undefined,
{ iceServers: [] },
{ ...QUALITY_PROFILES["720p30"], videoCodec },
);
await expect(peer.start()).resolves.toBe(true);
const preferences = FakePeerConnection.latest!.codecPreferenceCalls[0]!;
expect(preferences[0]?.mimeType.toLowerCase()).toBe(`video/${videoCodec}`);
expect(preferences.map(({ mimeType }) => mimeType.toLowerCase())).toEqual(
expect.arrayContaining(["video/h264", "video/vp8", "video/rtx"]),
);
},
);
it("applies STUN-only ICE configuration at creation and update", () => {
const peer = createPeer(
createStream(createTrack("video", "video"), null),
@@ -524,7 +537,7 @@ describe("HostPeer source replacement", () => {
expect(
connection.senders[0]?.setParameters.mock.calls.at(-1)?.[0],
).toMatchObject({
degradationPreference: "maintain-resolution",
degradationPreference: "balanced",
encodings: [{ maxBitrate: 8_000_000, maxFramerate: 60 }],
});
});
@@ -627,7 +640,7 @@ describe("HostPeer source replacement", () => {
expect(videoSender.setParameters).toHaveBeenCalledTimes(3);
expect(videoSender.setParameters.mock.calls.at(-1)?.[0]).toMatchObject({
degradationPreference: "maintain-resolution",
degradationPreference: "balanced",
encodings: [{ maxBitrate: 8_000_000, maxFramerate: 60 }],
});
});
+13 -1
View File
@@ -30,6 +30,11 @@ const qualitySettings = {
degradationPreference: "maintain-resolution",
} as const;
const qualitySettingsWithCodec = {
...qualitySettings,
videoCodec: "h264",
} as const;
const qualityEvidence = {
type: "viewer-quality-evidence",
guard: {
@@ -141,7 +146,7 @@ describe("client signaling protocol", () => {
expect(
clientMessageSchema.safeParse({
type: "authenticate",
protocol: "screener-v0",
protocol: "screener-v2",
roomId,
role: "viewer",
clientId: "client_12345678",
@@ -409,6 +414,12 @@ describe("client signaling protocol", () => {
qualitySettings,
}).success,
).toBe(true);
expect(
clientMessageSchema.safeParse({
type: "set-quality-settings",
qualitySettings: qualitySettingsWithCodec,
}).success,
).toBe(true);
for (const invalid of [
{ ...qualitySettings, resolution: "2160p" },
{ ...qualitySettings, maxFramerate: 14 },
@@ -418,6 +429,7 @@ describe("client signaling protocol", () => {
{ ...qualitySettings, maxBitrate: 12_000_001 },
{ ...qualitySettings, maxBitrate: 5_000_000.5 },
{ ...qualitySettings, degradationPreference: "automatic" },
{ ...qualitySettings, videoCodec: "vp9" },
{ ...qualitySettings, codec: "video/VP9" },
]) {
expect(
+16 -8
View File
@@ -6,7 +6,9 @@ import {
configureScreenAudioSender,
configureTwoLayerVideoSender,
configureVideoSender,
matchingQualityProfileId,
QUALITY_PROFILES,
qualitySettingsEqual,
SCREEN_AUDIO_MAX_BITRATE,
screenShareLowBitrate,
senderParameterWarning,
@@ -77,15 +79,21 @@ describe("realtime quality controls", () => {
});
});
it("defaults every recommended profile to clarity-first degradation", () => {
it("defaults every recommended profile to balanced degradation", () => {
expect(
Object.values(QUALITY_PROFILES).every(
(profile) =>
profile.degradationPreference === "maintain-resolution",
profile.degradationPreference === "balanced",
),
).toBe(true);
});
it("keeps codec selection orthogonal to the recommended quality profile", () => {
const h264 = { ...QUALITY_PROFILES["1080p60"], videoCodec: "h264" } as const;
expect(matchingQualityProfileId(h264)).toBe("1080p60");
expect(qualitySettingsEqual(h264, QUALITY_PROFILES["1080p60"])).toBe(false);
});
it("reads back every requested sender control after setParameters", async () => {
let applied = { encodings: [] } as unknown as RTCRtpSendParameters;
const setParameters = vi.fn(async (parameters: RTCRtpSendParameters) => {
@@ -109,14 +117,14 @@ describe("realtime quality controls", () => {
maxBitrate: 8_000_000,
maxFramerate: 60,
scaleResolutionDownBy: 4 / 3,
degradationPreference: "maintain-resolution",
degradationPreference: "balanced",
scalabilityMode: null,
},
applied: {
maxBitrate: 8_000_000,
maxFramerate: 60,
scaleResolutionDownBy: 4 / 3,
degradationPreference: "maintain-resolution",
degradationPreference: "balanced",
scalabilityMode: null,
},
mismatches: [],
@@ -146,7 +154,7 @@ describe("realtime quality controls", () => {
const before = { encodings: [{}] } as RTCRtpSendParameters;
const after = {
encodings: [{ maxBitrate: 8_000_000 }],
degradationPreference: "balanced",
degradationPreference: "maintain-resolution",
} as unknown as RTCRtpSendParameters;
const getParameters = vi
.fn<() => RTCRtpSendParameters>()
@@ -186,7 +194,7 @@ describe("realtime quality controls", () => {
scalabilityMode: "L1T2",
},
],
degradationPreference: "maintain-resolution",
degradationPreference: "balanced",
} as unknown as RTCRtpSendParameters;
const sender = {
track: { getSettings: () => ({ width: 1920, height: 1080 }) },
@@ -221,7 +229,7 @@ describe("realtime quality controls", () => {
},
{ rid: "high", scalabilityMode: "L1T2" },
],
degradationPreference: "maintain-resolution",
degradationPreference: "balanced",
} as unknown as RTCRtpSendParameters;
const sender = {
track: { getSettings: () => ({ width: 1920, height: 1080 }) },
@@ -261,7 +269,7 @@ describe("realtime quality controls", () => {
);
expect(applied).toMatchObject({
degradationPreference: "maintain-resolution",
degradationPreference: "balanced",
encodings: [
{
rid: "q",
+5 -4
View File
@@ -31,7 +31,8 @@ const defaultQualitySettings = {
resolution: "1080p",
maxFramerate: 60,
maxBitrate: 8_000_000,
degradationPreference: "maintain-resolution",
degradationPreference: "balanced",
videoCodec: "automatic",
} as const;
const balancedQualitySettings = {
resolution: "1080p",
@@ -6877,6 +6878,7 @@ describe("WebSocket signaling", () => {
oldClient.socket.send(
JSON.stringify({
type: "authenticate",
protocol: "screener-v2",
roomId: "999999999999",
role: "viewer",
clientId: "old-client",
@@ -6884,10 +6886,9 @@ describe("WebSocket signaling", () => {
);
expect(await oldClient.inbox.next("error")).toMatchObject({
code: "AUTH_REQUIRED",
message: "页面版本已更新,请刷新后重试",
code: "INVALID_MESSAGE",
});
expect(await closed).toEqual({ code: 4001, reason: "Protocol mismatch" });
expect(await closed).toEqual({ code: 1008, reason: "Invalid message" });
});
it("rejects an invalid upgrade request target without stopping the server", async () => {
+19 -27
View File
@@ -355,15 +355,6 @@ beforeEach(() => {
livekit.state.connectGate = null;
livekit.state.nextSenderParameterError = null;
vi.stubGlobal("MediaStream", FakeMediaStream);
vi.stubGlobal("RTCRtpSender", {
getCapabilities: vi.fn(() => ({
codecs: [
{ mimeType: "video/VP8", clockRate: 90_000 },
{ mimeType: "video/H264", clockRate: 90_000 },
],
headerExtensions: [],
})),
});
});
afterEach(() => {
@@ -445,7 +436,6 @@ describe("SfuPublisher", () => {
expect(room.localParticipant.publishTrack).toHaveBeenNthCalledWith(1, video, {
source: Track.Source.ScreenShare,
backupCodec: false,
videoCodec: "h264",
simulcast: true,
screenShareEncoding: {
maxBitrate: 8_000_000,
@@ -463,6 +453,9 @@ describe("SfuPublisher", () => {
],
degradationPreference: "maintain-resolution",
});
expect(
room.localParticipant.publishTrack.mock.calls[0]?.[1],
).not.toHaveProperty("videoCodec");
expect(room.localParticipant.publishTrack).toHaveBeenNthCalledWith(2, audio, {
source: Track.Source.ScreenShareAudio,
audioPreset: { maxBitrate: 128_000 },
@@ -507,24 +500,24 @@ describe("SfuPublisher", () => {
expect(room.disconnect).not.toHaveBeenCalled();
});
it("keeps VP8 for SFU publication when H.264 send capability is absent", async () => {
vi.stubGlobal("RTCRtpSender", {
getCapabilities: vi.fn(() => ({
codecs: [{ mimeType: "video/VP8", clockRate: 90_000 }],
headerExtensions: [],
})),
});
const publisher = new SfuPublisher();
await publisher.connect(connection);
it.each(["h264", "vp8"] as const)(
"passes an explicit %s preference to the SFU publisher",
async (videoCodec) => {
const publisher = new SfuPublisher();
await publisher.connect(connection);
await expect(
publisher.activate(stream(track("video", "video-1")), qualityProfile),
).resolves.toBe(true);
await expect(
publisher.activate(stream(track("video", "video-1")), {
...qualityProfile,
videoCodec,
}),
).resolves.toBe(true);
expect(
livekit.state.rooms[0].localParticipant.publications[0].options,
).toMatchObject({ videoCodec: "vp8", backupCodec: false });
});
expect(
livekit.state.rooms[0].localParticipant.publishTrack.mock.calls[0]?.[1],
).toMatchObject({ videoCodec, backupCodec: false });
},
);
it("fails closed when initial sender configuration is rejected", async () => {
const disconnected = vi.fn();
@@ -639,7 +632,6 @@ describe("SfuPublisher", () => {
expect(publication.options).toMatchObject({
source: Track.Source.ScreenShare,
backupCodec: false,
videoCodec: "h264",
screenShareEncoding: {
maxBitrate: 3_000_000,
maxFramerate: 30,