diff --git a/README.md b/README.md index 9624cf1..ce52984 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ Screener is a private, low-latency screen-sharing project for one game player and a small group of friends. Its media path is WebRTC P2P-first, with authenticated TURN fallback for network pairs that cannot connect directly. Viewers should be able to join from a desktop or mobile browser without installing the sender application. -The first measurable Web proof of concept is implemented. A host captures a screen, window, or browser tab and shares a numeric room code; friends can open `/r/{code}` directly or enter the code at `/join`, with no separate viewer token. Rooms are random and temporary by default. A password-protected deployment can optionally use SQLite-backed room numbers starting at `1`, reusable links, and a waiting state after sharing stops. Each viewer currently receives an independent WebRTC connection, so direct and TURN-relayed paths can coexist in one room. The default room capacity is eight viewers and deployments may configure 1 through 16, but that admission limit is not a performance promise. +The first measurable Web proof of concept is implemented. A host captures a screen, window, or browser tab and shares a numeric room code; friends can open `/r/{code}` directly or enter the code at `/join`, with no separate viewer token. During a share, the host can change between three ceiling profiles without reopening the source picker: 1080p60 at 8 Mbps, 1080p30 at 5 Mbps, and 720p30 at 3 Mbps. The host can also temporarily pause the picture or switch sources. Rooms are random and temporary by default. A password-protected deployment can optionally use SQLite-backed room numbers starting at `1`, reusable links, and a waiting state after sharing stops. Each viewer currently receives an independent WebRTC connection, so direct and TURN-relayed paths can coexist in one room. The default room capacity is eight viewers and deployments may configure 1 through 16, but that admission limit is not a performance promise. This is not yet a production release. Public TURN/UDP and TURN/TCP relay paths are verified, but real cross-network Screener media, game audio, a sustained 1:8 room, mobile browser lifecycle behavior, and latency targets still require the documented manual test matrix. diff --git a/docs/README.md b/docs/README.md index 91932e2..2618aa5 100644 --- a/docs/README.md +++ b/docs/README.md @@ -9,6 +9,7 @@ - [P2P WebRTC research](./research/webrtc-p2p-screen-sharing.md): evidence, bandwidth model, browser constraints, reference implementations, and feasibility assessment. - [Peer-assisted media research](./research/peer-assisted-media.md): browser shared-encode limits, standard relay re-encoding, rejected custom encoded transport, deterministic sticky topology, and abandon gates. - [Low-server-cost media routes](./research/low-server-media-routes.md): traffic accounting, route screening, conditional native encoded-RTP relay, and explicit SFU fallbacks. +- [Realtime quality adaptation](./research/realtime-quality-adaptation.md): browser degradation policy, live profile changes, offline-encoding boundaries, and measurement gates. - [Agent context governance research](./research/agent-context-governance.md): official Codex, Claude Code, Hermes Agent, and GitHub practices adopted by this repository. - [ADR-0001](./adr/0001-p2p-first-media-topology.md): accepted P2P-first media topology and its consequences. - [ADR-0002](./adr/0002-persistent-protected-rooms.md): optional SQLite persistence, protected sequential room IDs, and room lifecycle. diff --git a/docs/project-memory.md b/docs/project-memory.md index d48f40b..abc2890 100644 --- a/docs/project-memory.md +++ b/docs/project-memory.md @@ -9,36 +9,38 @@ Last updated: 2026-08-19 - Use a Photon-style central rendezvous service for access, rooms, signaling, deterministic topology, STUN, TURN fallback, and observability while clients carry media whenever practical. - Server bandwidth cost is a primary constraint. The media priority is direct P2P, then measured peer assistance, then an explicit user-operated or central SFU fallback. Never migrate topology silently. - The broadcaster has a hard target of at most two downstream media edges. Every assigned edge independently uses ICE and authenticated TURN/UDP or TURN/TCP when direct connectivity fails; P2P-first never means direct-only. -- Browser-only convenience does not override reachability, measurable performance, or honest resource accounting. Shared encoding reduces compute, not the network copy required by each viewer. +- Browser convenience does not override reachability, measurable performance, or honest resource accounting. Shared encoding reduces compute, not the last-hop copy required by every viewer. - Durable decisions, current snapshots, research, code, `AGENTS.md`, and `.codex/` belong in Git. Memory and status are rewritten in place rather than kept as transcripts. - Research current official sources and established implementations before material work. Apply Occam's razor and reject speculative protocols, services, scoring, and scale. ## Current Recommendation - Keep the deployed Windows Chrome/Edge broadcaster and responsive Web viewer as the measurement baseline. Validate current Android Chrome and iOS Safari as leaves. -- Use direct host P2P for one or two viewers. ADR-0004 owns the default-off experiment for later viewers: two sticky deterministic chains, host capacity two, viewer capacity one, no proactive rebalance or composite score, and no more than eight viewers when `PEER_ASSISTED_MEDIA=true`. +- Use direct host P2P for one or two viewers. Draft ADR-0004 and Draft PR #13 own the default-off experiment for later viewers: two sticky deterministic chains, host capacity two, viewer capacity one, no proactive rebalance or composite score, and no more than eight viewers when `PEER_ASSISTED_MEDIA=true`. - Standard browser relays resend remote `MediaStreamTrack` values and therefore decode and re-encode at every hop. Browser WebRTC does not guarantee one shared encoder across peer connections; measure this cost rather than hiding it. -- Every ADR-0004 gate is mandatory. Reject the browser experiment if it needs Encoded Transform/DataChannel/WebCodecs media, custom congestion or RTP recovery, multiple trees, relay scoring, transcoding, codec ladders, browser RTP injection, or more than two host edges. -- Only if topology, churn, connectivity, compatibility, and latency pass and re-encoding is the sole failure may a separate ADR propose a native shared-encode host plus opt-in native volunteer encoded-RTP relays. Any other failure closes peer assistance. +- Reject the browser experiment if it needs Encoded Transform/DataChannel/WebCodecs media, custom congestion or RTP recovery, multiple trees, relay scoring, transcoding, codec ladders, browser RTP injection, or more than two host edges. +- Only if every non-encoding gate passes and re-encoding is the sole failure may a separate ADR propose a native shared-encode host plus opt-in native volunteer encoded-RTP relays. Any other failure closes peer assistance. - Keep user-operated mini-SFU deployment and central single-node SFU Draft PR #12 as explicit fallbacks. They transfer fanout egress to their operator and do not justify automatic switching, Redis, or multi-node infrastructure. - Prefer direct UDP, then TURN/UDP, with TURN/TCP as the required non-UDP fallback. Optional TURN/TLS uses TCP 5349 by default; TCP 443 needs a dedicated address or validated L4/SNI routing. -- Treat 1080p60 as best effort and retain 720p60 and 720p30 fallbacks. Keep browser codec order until target hardware proves a more efficient common codec; do not add custom dynamic-FPS logic without stats showing a real gap. +- Treat 1080p60 at 8 Mbps as a best-effort ceiling. Keep the deployed 1080p60 at 8 Mbps, 1080p30 at 5 Mbps, and 720p30 at 3 Mbps profiles switchable during sharing. `balanced` does not guarantee resolution-first behavior; measure actual browser tradeoffs. +- Do not add custom scene detection or dynamic-FPS control until WebRTC statistics and host resource measurements prove a material gap. Keep browser codec order until target hardware proves a more efficient common codec. - Keep room policy deployment-driven. Public and password-only deployments use random temporary rooms. A site password plus SQLite path enables sequential persistent rooms; stopping sharing leaves the room and viewer link available. ## Current Implementation -- The repository is one npm package using Node.js 24, React, TypeScript, Vite, native WebRTC, `ws`, Zod, Vitest, and separate coturn. LAN/container listening defaults to all interfaces; bare-metal reverse proxy deployment binds loopback and exposes `/healthz`. -- The deployed media path still creates one independent host `RTCPeerConnection` per viewer. It supports capture-before-room creation, live source replacement, stable signaling identities, reconnect/ICE recovery, generation guards, short-lived TURN credentials, three manual quality profiles, and local WebRTC stats. It violates the new fanout target above two viewers. -- The spike branch contains deterministic server/client peer-assisted assignment and standard remote-track relay. It defaults off, is capped at eight viewers, re-encodes at every relay, and is neither merged nor deployed. Native shared encoding and custom encoded browser transport are not implemented. +- The repository is one npm package using Node.js 24, React, TypeScript, Vite, native WebRTC, `ws`, Zod, Vitest, and separate coturn. LAN/container listening defaults to all interfaces; bare-metal reverse-proxy deployment binds loopback and exposes `/healthz`. +- Production commit `5b2fb005f6f7` is deployed at `https://share.bonfire.icu`. It provides live source and quality changes without rebuilding healthy peers, video-only pause, clearer connection states, and the standard one-host-`RTCPeerConnection`-per-viewer path. Production therefore still exceeds the two-edge host target above two viewers. +- Production supports the three quality profiles above with `contentHint = "motion"` and explicit `degradationPreference = "balanced"`. A real live quality/pause cycle remains unverified. - Access remains deliberately small: optional site-wide `ACCESS_PASSWORD`, a 12-hour stateless HMAC HttpOnly `SameSite=Strict` cookie, internal host token, role-bound signaling, Origin/payload checks, and no accounts, JWTs, session map, or logout flow. -- With `ROOM_DATABASE_PATH` and the site password, built-in SQLite stores only auto-incremented room ID and host-token digest. Links persist and stopping sharing leaves viewers waiting. Without the path, rooms remain random and temporary; public mode cannot use sequential rooms. -- Commit `2f66770f8e90` is deployed at `https://share.bonfire.icu` behind nginx. Authenticated coturn at `turn.bonfire.icu:3478` has verified public STUN, TURN/UDP, TURN/TCP, and relay-only traffic. TURN/TLS is intentionally disabled. -- Chromium 151 has now demonstrated the intended three-viewer peer-assisted shape: exactly two connected host outbound peers; viewer 1 with one inbound plus one outbound forwarding to viewer 3; viewer 2 direct from the host; and decoded frames at all three viewers. Closing the first-level relay reattached its branch and resumed decoding in about 5.3 seconds while host active connected outbound edges peaked at two. -- A Chromium 151 one-to-eight synthetic functional smoke formed two depth-four chains: two connected host outbound peers, one inbound plus one outbound at viewers 1 through 6, inbound-only viewers 7 and 8, and decoded frames everywhere. This is short functional evidence, not proof of 720p60 quality, performance, latency, or endurance. -- That recovery covers only a page close immediately observed by the server. A silent partition waits for the 30-second heartbeat, so detection can take 30 to 60 seconds before the default 5-second viewer grace; it is unverified. The controlled server-observed recovery gate remains 5 seconds of grace plus at most 3 seconds to a decodable picture. -- Relay outbound stats are observable, but relay senders still use a fixed `1080p60`/8 Mbps envelope rather than the host-selected profile; profile propagation blocks valid comparative measurements. Real persistent-room reuse/restart, screen and game audio, heterogeneous networks, relay resource and generational-quality cost, depth-four latency, endurance, silent partition recovery, and mobile leaves remain unverified. -- On the current spike branch, `npm run check` passes type checking, 11 Vitest files with 129 tests, and both client and server production builds. -- Draft SFU PR #12 and ADR-0003 remain unmerged, undeployed, and independently reversible. There is no infrastructure blocker for the current Web deployment or bounded spike. +- With `ROOM_DATABASE_PATH` and the site password, built-in SQLite stores only auto-incremented room ID and host-token digest. Links persist and stopping sharing leaves viewers waiting. Production room ID `1` survived the `5b2fb005f6f7` deployment restart. Without the path, rooms remain random and temporary; public mode cannot use sequential rooms. +- Production runs behind nginx with Node.js 24.19.0. Authenticated coturn 4.17.2 at `turn.bonfire.icu:3478` has verified public STUN, TURN/UDP, TURN/TCP, and relay-only traffic. TURN/TLS is intentionally disabled. +- Draft PR #13 contains deterministic server/client peer-assisted assignment and standard remote-track relay. It defaults off, is capped at eight viewers, re-encodes at every relay, and is neither merged nor deployed. Native shared encoding and custom encoded browser transport are not implemented. +- Chromium 151 demonstrated the intended three-viewer shape: exactly two connected host outbound peers; viewer 1 with one inbound plus one outbound forwarding to viewer 3; viewer 2 direct from the host; and decoded frames at all three viewers. +- A Chromium 151 one-to-eight synthetic functional smoke formed two depth-four chains: two connected host outbound peers, one inbound plus one outbound at viewers 1 through 6, inbound-only viewers 7 and 8, and decoded frames everywhere. This proves topology only, not profile quality, resource cost, latency, or endurance. +- Closing the first-level relay reattached its branch and resumed decoding in about 5.3 seconds while host active connected outbound edges peaked at two. A silent partition instead waits for the 30-second heartbeat, so detection can take 30 to 60 seconds before the default 5-second viewer grace; that path is unverified. +- Relay outbound stats are observable, but relay senders still use a fixed `1080p60`/8 Mbps envelope rather than the host-selected profile. Profile propagation blocks valid comparative quality and resource measurements. +- The recorded peer-assisted full check passes type checking, 12 Vitest files with 135 tests, and both client and server production builds. Production release `5b2fb005f6f7` separately passed 108 Vitest tests, both builds, loopback and public HTTPS/access-gate checks, and a clean activation without disturbing nginx, coturn, or the blog. +- Draft SFU PR #12 and ADR-0003 remain unmerged, undeployed, and independently reversible. There is no infrastructure blocker for the deployed Web baseline or bounded peer-assisted experiment. ## Provisional Quality Targets @@ -51,7 +53,7 @@ These are measurement gates, not performance claims. ## Open Decisions -- Sustainable viewer count by broadcaster hardware, quality profile, network class, and media route; use instrumented 1/3/5/8 comparisons rather than a 1:8 claim. +- Sustainable viewer count by broadcaster and relay hardware, quality profile, network class, and media route; use instrumented 1/3/5/8 comparisons rather than a 1:8 claim. - Whether ADR-0004 passes fanout, re-encoding, depth-four latency, reparenting, silent-partition, and mobile-leaf gates. - Exact mobile lifecycle behavior and whether Windows per-application audio is required for the first release. - Project license and distribution model, which determines whether GPL/AGPL sources can move beyond study-only use. @@ -60,9 +62,8 @@ These are measurement gates, not performance claims. ## Source Of Truth -- Documentation index: `docs/README.md` -- Current phase: `docs/status.md` -- Requirements and first PoC design: linked from the documentation index -- Media research: `docs/research/webrtc-p2p-screen-sharing.md`, `docs/research/peer-assisted-media.md`, and `docs/research/low-server-media-routes.md` +- Documentation index and current phase: `docs/README.md` and `docs/status.md` +- Requirements and design: `docs/需求理解.md` and `docs/方案设计.md` +- Media research: `docs/research/webrtc-p2p-screen-sharing.md`, `docs/research/realtime-quality-adaptation.md`, `docs/research/peer-assisted-media.md`, and `docs/research/low-server-media-routes.md` - Architecture: ADR-0001, ADR-0002, proposed experiment ADR-0004, and Draft PR #12's unaccepted ADR-0003 - Deployment and maintenance: `docs/deployment.md`, `docs/maintenance.md`, `AGENTS.md`, and `.codex/` diff --git a/docs/research/realtime-quality-adaptation.md b/docs/research/realtime-quality-adaptation.md new file mode 100644 index 0000000..06d0cd4 --- /dev/null +++ b/docs/research/realtime-quality-adaptation.md @@ -0,0 +1,121 @@ +# Realtime Screen-Share Quality Adaptation + +- Research date: 2026-08-19 +- Scope: realtime game screen sharing in the browser +- Status: implementation input; real-device quality remains unverified + +## Finding + +Screener should not optimize a single quality dimension at the expense of the +others. The previous combination of `contentHint = "motion"` and an explicit +`degradationPreference = "maintain-framerate"` told WebRTC to preserve frame +rate primarily by reducing resolution. That is appropriate for some fast +motion, but it can make game UI, maps, subtitles, and text unreadable. + +The smallest correction is to retain the game-oriented `motion` hint while +explicitly selecting `balanced` degradation. The WebRTC API allows the user +agent to balance frame-rate and resolution degradation; it does not prescribe +the algorithm or guarantee a clarity-first result. Congestion control remains +authoritative, and Chromium, Firefox, and Safari behavior must be measured. + +The inspected Chromium/libwebrtc source chain makes a screen-only shortcut +especially unsafe to assume for Screener. The JavaScript `motion` hint reaches +`VideoRtpSender` as `kFluid`, which explicitly clears `options.is_screencast`. +`WebRtcVideoSendStream` then configures the encoder as `kRealtimeVideo`, while +`VideoStreamEncoder` only reinterprets `BALANCED` as `MAINTAIN_RESOLUTION` when +its encoder is marked as screen share. The explicit `balanced` preference still +wins the earlier preference-selection branch, but the later screen-only remap +does not follow from this project path. Treat the effective tradeoff as an +implementation detail to verify, not a product guarantee. + +An older libwebrtc experiment could detect animated screen content under +`balanced` and cap it to 1280x720. Commit `1d7d0e6e2c50` deleted that experiment +on 2024-05-22; the commit says it had already been disabled for several years +and was not maintained. Current Chromium behavior must therefore not be +described as automatically detecting game motion and forcing a 720p cap. + +## Why Offline Encoding Presets Do Not Transfer + +[Maruko Toolbox](https://github.com/wzxjohn/marukotoolbox) is an Apache-2.0 +GUI around offline video-processing tools. Its x264-style CRF, slow presets, +lookahead, and multi-pass workflows trade encoding time and buffering for file +size and quality. Interactive WebRTC cannot buffer or revisit future frames in +that way. The transferable principle is to treat bitrate, image detail, frame +rate, and encode cost as a joint tradeoff; its concrete offline presets are not +a WebRTC configuration baseline. +The GUI repository is Apache-2.0, while x264 itself uses GPL-2.0 or commercial +licensing; no implementation or preset is copied into Screener. + +Discord's published Go Live design reaches the same broad conclusion for +realtime game streaming: bandwidth estimates, frame delivery, image quality, +latency, CPU, and memory conflict and must be balanced. Its native encoder +tuning and hardware integration are not available to a browser-only sender and +must not be presented as settings this project already has. + +## Current Policy + +The three user-visible profiles remain ceilings rather than promised rates: + +| Profile | Capture ceiling | Sender bitrate ceiling | +| --- | --- | ---: | +| 1080p60 | 1920x1080 at 60 fps | 8 Mbps | +| 1080p30 | 1920x1080 at 30 fps | 5 Mbps | +| 720p30 | 1280x720 at 30 fps | 3 Mbps | + +The middle profile is an explicit manual 1080p30 ceiling: choosing it trades a +60 fps ceiling for a 1080p capture bound. LiveKit currently uses the same +1080p30 at 5 Mbps screen-share preset, but neither preset guarantees the emitted +resolution, frame rate, or bitrate. + +- Initial capture supplies `ideal` and `max` bounds for resolution and frame + rate; the actual result is read back from `MediaStreamTrack.getSettings()`. +- A live profile change uses `track.applyConstraints()` and updates every + current sender with `RTCRtpSender.setParameters()`. It does not reopen the + source picker or renegotiate healthy peer connections. +- The video track keeps `contentHint = "motion"`, while each sender explicitly + uses `degradationPreference = "balanced"`. This is a user-agent preference, + not a promise to preserve either resolution or frame rate. +- `maxBitrate` and `maxFramerate` are ceilings. They are neither minimums nor + target guarantees, and the project does not use SDP bitrate hacks. +- Pausing the picture disables the existing video track, producing black video + without closing the room or media connection. Audio remains enabled. + +## Deliberate Non-Goals + +- No canvas pixel-difference detector, machine-learned rate controller, or + periodic profile switching. +- No copied x264 CRF/preset recipe in the browser path. +- No forced codec order until target hardware measurements identify the actual + power-efficient encoder. +- No promise that all browsers honor `applyConstraints` or degradation + preference identically. + +## Verification Gate + +Use the same static UI scene and deterministic high-motion game scene at +720p30, 1080p30, and 1080p60. Record actual capture and outbound dimensions/fps, +bitrate, `qualityLimitationReason`, codec, encoder implementation, encode time, +dropped frames, RTT, packet loss, and host CPU/GPU utilization. A live profile +change must preserve peer connection IDs, avoid a second source prompt, and +visibly converge to the requested bounds. + +Compare image readability and motion continuity instead of declaring success +from FPS alone. If reproducible evidence later shows that `balanced` still +oscillates or makes the wrong tradeoff on supported machines, revise the fixed +profiles before adding an application-level adaptation controller. + +## Primary Sources + +- [W3C MediaStreamTrack Content Hints](https://www.w3.org/TR/mst-content-hint/) +- [W3C Screen Capture](https://www.w3.org/TR/screen-capture/) +- [W3C WebRTC](https://www.w3.org/TR/webrtc/) +- [Chromium `motion` to libwebrtc `kFluid` bridge](https://chromium.googlesource.com/chromium/src/third_party/+/refs/heads/main/blink/renderer/modules/peerconnection/media_stream_video_webrtc_sink.cc) +- [libwebrtc `motion`/`kFluid` sender classification](https://webrtc.googlesource.com/src/+/3b1eab8a69cb5078befb021c5492d3f204a7d6a2/pc/rtp_sender.cc) +- [libwebrtc encoder content-type selection](https://webrtc.googlesource.com/src/+/9caef2a8b88f389af10cee841732c42a98d3d45d/media/engine/webrtc_video_engine.cc) +- [libwebrtc conditional screen-share degradation mapping](https://webrtc.googlesource.com/src/+/refs/heads/main/video/video_stream_encoder.cc) +- [libwebrtc removal of the automatic animation-detection experiment, 2024-05-22](https://webrtc.googlesource.com/src/+/1d7d0e6e2c5002815853be251ce43fe88779ac85) +- [LiveKit screen-share presets](https://github.com/livekit/client-sdk-js/blob/main/src/room/track/options.ts) +- [LiveKit degradation defaults](https://github.com/livekit/client-sdk-js/blob/main/src/room/participant/publishUtils.ts) +- [Jitsi desktop degradation preference](https://github.com/jitsi/lib-jitsi-meet/blob/master/modules/RTC/TraceablePeerConnection.ts) +- [Discord Go Live architecture](https://discord.com/blog/how-it-all-goes-live-an-overview-of-discords-streaming-technology) +- [Discord encoder-quality case study](https://discord.com/blog/from-blocky-to-brilliant-improving-video-quality-on-discord-go-live-on-amd-gpus) diff --git a/docs/research/webrtc-p2p-screen-sharing.md b/docs/research/webrtc-p2p-screen-sharing.md index aa98d2c..f993340 100644 --- a/docs/research/webrtc-p2p-screen-sharing.md +++ b/docs/research/webrtc-p2p-screen-sharing.md @@ -154,11 +154,11 @@ const parameters = sender.getParameters(); parameters.encodings ??= [{}]; parameters.encodings[0].maxFramerate = 60; parameters.encodings[0].maxBitrate = selectedProfileBitrate; -parameters.degradationPreference = "maintain-framerate"; +parameters.degradationPreference = "balanced"; await sender.setParameters(parameters); ``` -这些参数是偏好或上限，不能绕过浏览器拥塞控制，也不能保证目标码率。Screen Capture 和 WebRTC 规范允许浏览器决定实际输出帧率、编码与拥塞行为；`contentHint = "motion"` 也只是内容提示。静态画面通常能降低编码数据量，但规范不保证所有浏览器主动降低捕获频率或 GPU 开销。 +这些参数是偏好或上限，不能绕过浏览器拥塞控制，也不能保证目标码率。`balanced` 允许浏览器在分辨率和帧率之间权衡，但不规定具体算法；本项目的 `contentHint = "motion"` 还会影响 Chromium/libwebrtc 对来源的内部分类，因此不能套用其仅限 screen encoder 的 `maintain-resolution` 重解释。当前源码链与测量要求见 [Realtime quality adaptation](./realtime-quality-adaptation.md)。静态画面通常能降低编码数据量，但规范不保证所有浏览器主动降低捕获频率或 GPU 开销。 因此首版不实现画面差分检测、周期性 `applyConstraints()` 或自定义动态 FPS 状态机。先对静态桌面和高动态游戏分别记录 `framesEncoded`、发送码率、`totalEncodeTime / framesEncoded`、`qualityLimitationReason` 及主机 CPU/GPU 占用；只有测量显示浏览器行为留下显著问题时，再设计最小的控制策略。这避免用额外竞态、计时器和画质跳变解决一个可能已由捕获器、编码器和 WebRTC 拥塞控制处理的问题。 diff --git a/docs/status.md b/docs/status.md index bca2551..2899d68 100644 --- a/docs/status.md +++ b/docs/status.md @@ -4,74 +4,73 @@ Last updated: 2026-08-19 ## Phase -The standard one-`RTCPeerConnection`-per-viewer WebRTC PoC is deployed at -`https://share.bonfire.icu`. Commit `2f66770f8e90` is the deployed baseline for -protected SQLite rooms, sequential reusable IDs, and waiting after sharing -stops. This path still exceeds the new host-fanout target above two viewers. +The WebRTC proof of concept is deployed at `https://share.bonfire.icu`. Commit +`5b2fb005f6f7` adds live 1080p60/1080p30/720p30 quality changes, video-only +pause, clearer connection states, protected SQLite persistence, sequential room +IDs, reusable links, and a waiting state after sharing stops. Production room ID +`1` survived the deployment restart. -The spike branch contains a default-off, standard-WebRTC peer-assisted -experiment for later viewers. It assigns two sticky chains with host capacity -two and viewer capacity one, then relays remote tracks with a decode/re-encode -at every browser hop. `PEER_ASSISTED_MEDIA=false` remains the default and the -mode cannot be enabled above eight viewers. Limited Chromium evidence now proves -the intended three-viewer shape, a synthetic eight-viewer functional topology, -and one controlled page-close recovery, but the mode is not merged, deployed, -or production-validated. Draft SFU PR #12 is also unmerged and undeployed. +Production still uses one host `RTCPeerConnection` per viewer. Draft PR #13 adds +a default-off, standard-WebRTC peer-assisted experiment for later viewers: two +sticky chains with host capacity two, viewer capacity one, and decode/re-encode +at every browser relay hop. `PEER_ASSISTED_MEDIA=false` remains the default, the +mode cannot be enabled above eight viewers, and it is neither merged, deployed, +nor production-validated. Draft SFU PR #12 is also unmerged and undeployed. ## Current Snapshot - Product scope is private game sharing for one broadcaster and a small trusted friend group; public broadcasting remains out of scope. - Runtime is Node.js 24, React, TypeScript, Vite, native browser WebRTC, `ws`, Zod, Vitest, and a separate coturn deployment. -- Capture precedes room creation. Healthy peers survive source replacement, and local stats report path, candidate, RTT, bitrate, frame, loss, jitter, codec, and quality-limitation data. +- Capture precedes room creation. Live source and quality changes preserve healthy peers; picture pause keeps audio and connections active. `contentHint = "motion"` and explicit `balanced` degradation do not guarantee resolution-first behavior. - Whole-site `ACCESS_PASSWORD` is optional. Protected sessions use a stateless 12-hour HMAC HttpOnly `SameSite=Strict` cookie; host authentication remains internal and signaling is role-bound. - Without `ROOM_DATABASE_PATH`, rooms are random and temporary. With both the database path and site password, room IDs start at `1`, links persist, and stopping a share leaves viewers waiting. SQLite stores only room ID and host-token digest. - Direct ICE is preferred independently per media edge. Authenticated TURN/UDP and TURN/TCP are required production fallbacks; TURN/TLS is optional. -- The experimental media priority remains direct P2P for one or two viewers, peer-assisted only after its gates pass, then user-operated or central single-node SFU as an explicit fallback. There is no automatic migration. +- The media priority is direct P2P for one or two viewers, peer assistance only after every ADR-0004 gate passes, then user-operated or central single-node SFU as an explicit fallback. There is no automatic migration. ## Verified Evidence -- On the current spike branch, `npm run check` passes type checking, 11 Vitest files with 129 tests, and both client and server production builds. -- The production site runs behind nginx on Debian 12 with Node.js 24.19.0. Authenticated coturn 4.17.2 serves public STUN and TURN/UDP+TCP on `turn.bonfire.icu:3478`; TURN/TLS is intentionally disabled. -- HTTPS/WSS, the site gate and cookie, room/WebSocket authorization, certificate renewal, SQLite permissions and clean service restart are verified. Public Chromium relay-only connections passed bidirectional traffic over TURN/UDP and TURN/TCP. -- Earlier Chromium 151 synthetic-media checks covered direct UDP media, offer/answer/create-offer recovery, signaling reconnect generation isolation, live video/audio source replacement, and the protected 390 px viewer flow. -- In a real Chromium 151 one-to-three-viewer run, the host held exactly two connected outbound media peers. Viewer 1 held one inbound and one outbound peer and forwarded media to viewer 3; viewer 2 remained directly attached to the host. All three viewers decoded frames. -- A Chromium 151 one-to-eight synthetic functional smoke formed two depth-four chains. The host held two connected outbound peers; viewers 1 through 6 each held one inbound and one outbound peer; viewers 7 and 8 were inbound-only leaves; all viewers decoded frames. This short smoke proves functional topology only, not 720p60 quality, resource cost, latency, or endurance. -- Closing that first-level relay page caused its branch to reattach and decode again after about 5.3 seconds. The host's peak active connected outbound media-edge count remained two. -- That recovery proves only the path where the server observes a page close immediately. A silent network partition depends on the 30-second heartbeat, so detection can take 30 to 60 seconds before the 5-second viewer grace begins; that path is unverified. +- Production release `5b2fb005f6f7` passed type checking, 108 Vitest tests, both production builds, loopback smoke, public HTTPS/access-gate checks, and clean activation under `/opt/screener/current`. The new 1080p30 asset loaded, Screener had zero automatic restarts, and nginx, coturn, and the existing blog remained healthy. +- The protected production database and permissions survived restart, and room ID `1` remained present. Stop-and-republish and link reuse still require a browser cycle. +- The recorded peer-assisted full check passes type checking, 12 Vitest files with 135 tests, and both client and server production builds. +- Chromium 151 one-to-three evidence held exactly two connected host outbound peers. Viewer 1 held one inbound plus one outbound peer and forwarded to viewer 3; viewer 2 stayed direct; all three decoded frames. +- A Chromium 151 one-to-eight synthetic functional smoke formed two depth-four chains. The host held two outbound peers; viewers 1 through 6 each held one inbound plus one outbound; viewers 7 and 8 were leaves; all decoded frames. This proves topology only, not quality, resource cost, latency, or endurance. +- Closing the first-level relay caused its branch to reattach and decode again after about 5.3 seconds while host active connected outbound edges peaked at two. +- Production HTTPS/WSS, access cookie, room/WebSocket authorization, certificate renewal, public STUN, and authenticated TURN/UDP and TURN/TCP relay-only bidirectional paths are verified. TURN/TLS is intentionally disabled. +- Headless Chromium 151 synthetic checks cover direct UDP media, offer/answer/create-offer recovery, generation isolation, live video/audio replacement, quality parameter updates, and video-only pause. Layout checks used actual inner widths 500, 781, and 820 px; a true 390 px viewport remains unverified. ## Unverified Boundaries -- The peer-assisted mode has not completed 1/3/5/8-viewer 30-minute runs, relay CPU/GPU and generational-quality measurements, controlled loss/RTT tests, or depth-four latency gates. -- Android Chrome and iOS Safari remain required leaves but are not verified for this topology. The protocol has no relay-capability bit; controlled join order is the only mobile-leaf enforcement, so arbitrary-user deployment is excluded. -- Real screen/game audio, heterogeneous machines and networks, browser lifecycle behavior, and sustained 1080p60 or 720p60 performance remain unverified. -- The first real protected room still needs room ID `1`, stop-and-republish, link reuse, and service-restart verification. -- Shared browser encoding is not available. Encoded Transform, DataChannel/WebCodecs media, custom congestion control, multiple trees, relay scoring, and automatic SFU switching remain outside the spike. -- Relay outbound diagnostics are visible, but relays still use a fixed `1080p60`/8 Mbps sender envelope instead of the host's selected profile. Profile propagation must be unified before quality or resource comparisons are valid. +- Relay senders still use a fixed `1080p60`/8 Mbps envelope instead of the host-selected room profile. Profile propagation blocks valid peer-assisted quality and resource comparisons. +- Peer assistance has not completed 1/3/5/8-viewer 30-minute runs, relay CPU/GPU and generational-quality measurements, controlled loss/RTT tests, or depth-four latency gates. +- Android Chrome and iOS Safari remain required leaves but are not verified for this topology. There is no runtime relay-capability bit; controlled join order is the only mobile-leaf enforcement, so arbitrary-user deployment is excluded. +- The observed recovery starts from a page close immediately seen by the server. A silent partition can wait 30 to 60 seconds for heartbeat detection before the default 5-second grace; it remains unverified. +- Real screen/game audio, heterogeneous machines and networks, mobile lifecycle behavior, the production live quality/pause cycle, room `1` stop-and-republish/link reuse, and sustained profile performance remain unverified. +- Browser relays do not provide shared encoding. Encoded Transform, DataChannel/WebCodecs media, custom congestion control, multiple trees, relay scoring, and automatic SFU switching remain outside the experiment. ## Next Milestone -Complete only the bounded ADR-0004 experiment: +Complete only the bounded ADR-0004 experiment, with distribution work ahead of +UI polish: -- preserve host fanout at two and viewer fanout at one across joins, reconnects, grace expiry, and reparenting; - propagate one room quality profile to host and relay senders before comparative media measurements; -- run reproducible 1/3/5/8-viewer 720p60 measurements for topology depth, selected ICE path, host/relay upload, encode/decode work, first picture, decoded FPS, quality limitation, and glass-to-glass latency; -- repeat first-level relay loss with server-observed close and separately measure silent heartbeat-detected partition recovery; and +- preserve host fanout at two and viewer fanout at one across joins, reconnects, grace expiry, and reparenting; +- run reproducible 1/3/5/8-viewer measurements using the deployed profiles, recording topology depth, selected ICE path, host/relay upload, encode/decode work, first picture, decoded FPS, quality limitation, and glass-to-glass latency; +- repeat first-level relay loss with a server-observed close and separately measure silent heartbeat-detected partition recovery; and - verify current Chrome/Edge relays plus Android Chrome and iOS Safari leaves. -For a disconnect immediately observed by the server, the recovery gate retains -the default 5-second grace followed by at most 3 seconds to restore a decodable -picture, about 8 seconds total. Silent partitions must be reported separately -with their heartbeat-detection delay. +For a disconnect immediately observed by the server, the gate retains the +default 5-second grace followed by at most 3 seconds to restore a decodable +picture, about 8 seconds total. Silent partitions include their detection delay. All gates fail closed. If success requires a custom browser media plane, changed RTP recovery, multiple trees, scoring, transcoding, a codec ladder, or more than -two host edges, remove this experiment. Only an isolated re-encoding failure may +two host edges, remove the experiment. Only an isolated re-encoding failure may advance to a separate native shared-encode and volunteer encoded-RTP proposal. ## Blockers And Decisions -No infrastructure blocker remains. Production adoption is blocked on the -measurement matrix above, not implementation claims. +No infrastructure blocker remains. Production adoption of peer assistance is +blocked on profile propagation and the measurement matrix above. - Whole-system versus selected-game audio for the first release. - Initial deployment region and network cohort. diff --git a/docs/方案设计.md b/docs/方案设计.md index a1d16a0..7cf1d8b 100644 --- a/docs/方案设计.md +++ b/docs/方案设计.md @@ -130,10 +130,10 @@ Zod 在服务边界拒绝未知类型、超长字符串、非法 SDP/candidate | 档位 | 约束目标 | 初始码率上限 | | --- | --- | ---: | | 1080p60 | 1920x1080、60 fps | 8 Mbps/peer | -| 720p60 | 1280x720、60 fps | 5 Mbps/peer | +| 1080p30 | 1920x1080、30 fps | 5 Mbps/peer | | 720p30 | 1280x720、30 fps | 3 Mbps/peer | -浏览器可返回更低设置，界面以 `track.getSettings()` 显示实际结果。视频 track 设置 `contentHint = "motion"`；sender 尝试设置 `maxBitrate`、`maxFramerate` 与 `degradationPreference = "maintain-framerate"`，不支持时只记录警告。 +浏览器可返回更低设置，界面以 `track.getSettings()` 显示实际结果。视频 track 保留 `contentHint = "motion"`；sender 尝试设置 `maxBitrate`、`maxFramerate` 与 `degradationPreference = "balanced"`。`balanced` 只允许浏览器在帧率和分辨率之间权衡，不保证清晰度优先；具体 Chromium、Firefox 与 Safari 行为仍需实测。不支持 sender 参数时只记录警告。 PoC 不强制 codec，也不做 SDP munging。先通过 stats 记录实际协商 codec、`encoderImplementation` 和 `powerEfficientEncoder`，再依据目标机器测量决定是否优先 H.264。这样避免静态 codec 偏好意外触发软件编码。 @@ -141,6 +141,8 @@ PoC 不强制 codec，也不做 SDP munging。先通过 stats 记录实际协商 直播中切换来源仍由明确的按钮手势重新调用 `getDisplayMedia()`，不改变房间、host generation 或媒体拓扑。选源取消或失败时继续发送旧流；成功后为每个现有 sender 调用 `replaceTrack()`，健康连接不重新协商。初始 offer 预留 send-only audio transceiver，使音频可以在 `null -> track -> null` 间切换。若某个 peer 的替换或回滚失败，只重建该 peer；新流提交后才停止旧流，并以当前 stream 身份保护旧 track 的迟到 `ended` 事件。统计码率累加器在换源后重置，避免把不同 track 的计数器混算。 +直播中切换质量档位不重新打开选源器：对当前视频 track 调用 `applyConstraints()`，再更新所有现有 sender 的参数；新加入或重建的 sender 使用当前档位。失败时保留当前分享并给出状态，不以重建整个房间补偿。临时暂停只禁用当前视频 track，使远端收到黑画面；音频、房间和连接保持不变，恢复时重新启用同一 track。 + `frameRate` 约束和 sender 的 `maxFramerate` 都是目标或上限；浏览器仍控制实际捕获、重复帧处理、编码和拥塞降级。静态画面可能由浏览器减少实际编码数据，但标准没有保证每个实现以同一种方式节省采集或 GPU 开销。本 PoC 不添加画面差分、定时改约束或自定义动态 FPS 状态机，先用 `framesEncoded`、实际发送码率、`totalEncodeTime`、质量受限原因和主机资源占用测量真实问题。 自动全房间降档同样需要可信的聚合策略，放在测量后的下一阶段；本 PoC 提供手动档位和明确的受限原因，不伪装成已经解决自适应问题。 diff --git a/docs/需求理解.md b/docs/需求理解.md index b05894c..4f0904b 100644 --- a/docs/需求理解.md +++ b/docs/需求理解.md @@ -14,13 +14,13 @@ - [ ] 简单房间：分享者获得纯数字房间码和 `/r/{code}` 链接；观看者也可在 `/join` 只输入房间码加入，链接不携带额外观看 token 或 fragment。启用持久受保护房间时，房间码从 `1` 开始自增；其他模式继续使用随机临时房间码防止枚举。 - [ ] 免安装观看：桌面和手机观看者可直接通过浏览器链接进入、授权并播放，不依赖与分享端相同的桌面客户端。 -- [ ] 屏幕采集：分享者可主动选择显示器或应用窗口，开始、停止和切换分享状态。 +- [ ] 屏幕采集：分享者可主动选择显示器或应用窗口，开始、停止、临时暂停画面和切换分享来源。 - [ ] 游戏声音：Windows Chrome/Edge 目标环境可请求系统音频；没有返回音轨时必须明确提示，而不是静默无声。 - [ ] 一对多观看：一名分享者可向配置允许的多名观看者发送画面；默认上限为 8，可配置范围为 1 至 16。当前 PoC 的逐观看者连接是测量基线，不再是超过两名观看者时的目标拓扑。 - [ ] 分享端 fanout 硬上限：分享端任何时刻最多维护两条下游媒体边，包括加入、断线、换父和恢复过程。第三名及后续观看者只能由通过验收的 peer-assisted 拓扑、部署者显式选择的可选 SFU 承载，或被明确拒绝；不能隐藏地回退为分享端第三条连接。 - [ ] P2P 优先：每条逻辑媒体边的 ICE 必须先尝试直接 UDP 路径，服务器不得默认中转可直连的媒体。一个或两个观看者优先直接连接分享端；更多观看者的 peer-assisted 行为须先通过 ADR-0004 的实验门槛。 - [ ] NAT 兜底：每条 parent-child 边独立进行 ICE；生产环境必须提供 STUN、TURN/UDP 与 TURN/TCP candidates，通过优先级偏好直连和 UDP，同时允许无法使用 UDP 的网络选择 TCP，不能因为某条边的网络与其他边不同而静默失败。 -- [ ] 质量档位：至少提供 1080p60、720p60、720p30 三档目标，并依赖 WebRTC 拥塞控制及明确的手动档位避免持续排队；自定义动态 FPS 策略必须在统计证明有必要后再实现。 +- [ ] 质量档位：提供 1080p60（8 Mbps）、1080p30（5 Mbps）、720p30（3 Mbps）三档上限；分享期间可直接换档而不重新选择来源或重建健康连接。保留 `contentHint = "motion"` 并显式设置 `degradationPreference = "balanced"`；该偏好允许浏览器在帧率和分辨率之间权衡，不保证清晰度优先，具体行为必须实测。自定义动态 FPS 或画面检测策略必须在统计证明有必要后再实现。 - [ ] 断线恢复：临时网络变化时尝试 ICE restart 和重连，并向用户显示当前连接状态。 - [ ] 可观测性：采集候选类型、RTT、丢包、抖动、实际码率、帧率、编码耗时和质量受限原因。 - [ ] 访问控制：部署者可配置一个全站 `ACCESS_PASSWORD`；为空时网站公开，非空值必须为 1 至 128 个可见 ASCII 字符。配置时分享者和观看者都必须先登录。登录成功签发 12 小时、无服务端会话状态的 HttpOnly、`SameSite=Strict` cookie，建房 HTTP 和 WebSocket 都只使用该 cookie，不提供建房 Bearer 旁路；不引入账号、JWT、服务端 session Map 或 logout。 diff --git a/src/client/media/quality.ts b/src/client/media/quality.ts index 00c759d..2a05058 100644 --- a/src/client/media/quality.ts +++ b/src/client/media/quality.ts @@ -6,11 +6,11 @@ export const QUALITY_PROFILES = { frameRate: 60, maxBitrate: 8_000_000, }, - "720p60": { - label: "720p 60", - width: 1280, - height: 720, - frameRate: 60, + "1080p30": { + label: "1080p 30", + width: 1920, + height: 1080, + frameRate: 30, maxBitrate: 5_000_000, }, "720p30": { @@ -25,6 +25,14 @@ export const QUALITY_PROFILES = { export type QualityProfileId = keyof typeof QUALITY_PROFILES; export type QualityProfile = (typeof QUALITY_PROFILES)[QualityProfileId]; +function captureConstraints(profile: QualityProfile): MediaTrackConstraints { + return { + width: { ideal: profile.width, max: profile.width }, + height: { ideal: profile.height, max: profile.height }, + frameRate: { ideal: profile.frameRate, max: profile.frameRate }, + }; +} + export async function captureDisplay( profile: QualityProfile, ): Promise { @@ -33,11 +41,7 @@ export async function captureDisplay( } const stream = await navigator.mediaDevices.getDisplayMedia({ - video: { - width: { ideal: profile.width }, - height: { ideal: profile.height }, - frameRate: { ideal: profile.frameRate, max: profile.frameRate }, - }, + video: captureConstraints(profile), audio: true, }); @@ -50,6 +54,26 @@ export async function captureDisplay( return stream; } +export async function applyCaptureProfile( + stream: MediaStream, + profile: QualityProfile, +): Promise { + const videoTrack = stream.getVideoTracks()[0]; + if (!videoTrack) { + throw new Error("共享流缺少视频轨道"); + } + await videoTrack.applyConstraints(captureConstraints(profile)); +} + +export function setVideoPaused(stream: MediaStream, paused: boolean): boolean { + const videoTrack = stream.getVideoTracks()[0]; + if (!videoTrack) { + return false; + } + videoTrack.enabled = !paused; + return true; +} + export async function configureVideoSender( sender: RTCRtpSender, profile: QualityProfile, @@ -60,6 +84,6 @@ export async function configureVideoSender( } parameters.encodings[0].maxBitrate = profile.maxBitrate; parameters.encodings[0].maxFramerate = profile.frameRate; - parameters.degradationPreference = "maintain-framerate"; + parameters.degradationPreference = "balanced"; await sender.setParameters(parameters); } diff --git a/src/client/pages/HostPage.tsx b/src/client/pages/HostPage.tsx index 12c5ceb..8dc38c0 100644 --- a/src/client/pages/HostPage.tsx +++ b/src/client/pages/HostPage.tsx @@ -3,6 +3,8 @@ import { Copy, Hash, MonitorUp, + Pause, + Play, RefreshCw, Square, Users, @@ -31,8 +33,10 @@ import { } from "../lib/session"; import { SignalingClient } from "../lib/signaling"; import { + applyCaptureProfile, captureDisplay, QUALITY_PROFILES, + setVideoPaused, type QualityProfileId, } from "../media/quality"; import type { @@ -120,6 +124,8 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { const [notice, setNotice] = useState(null); const [copied, setCopied] = useState(false); const [switchingSource, setSwitchingSource] = useState(false); + const [changingQuality, setChangingQuality] = useState(false); + const [picturePaused, setPicturePaused] = useState(false); const videoRef = useRef(null); const streamRef = useRef(null); @@ -130,6 +136,9 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { const generationRef = useRef(0); const activeGenerationRef = useRef(null); const sourceSwitchRef = useRef(null); + const qualityChangeRef = useRef(null); + const qualityIdRef = useRef("1080p60"); + const picturePausedRef = useRef(false); const retiringStreamRef = useRef(null); const viewers = useMemo( @@ -148,6 +157,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { activeGenerationRef.current = null; generationRef.current += 1; sourceSwitchRef.current = null; + qualityChangeRef.current = null; signalRef.current?.stop(); peersRef.current.forEach((peer) => peer.dispose()); peersRef.current.clear(); @@ -168,6 +178,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { function disposeResources(notifyServer: boolean): void { sourceSwitchRef.current = null; + qualityChangeRef.current = null; const signal = signalRef.current; if (signal) { if (notifyServer) { @@ -193,6 +204,9 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { setPeerSnapshots(new Map()); setSignalStatus("offline"); setSwitchingSource(false); + setChangingQuality(false); + picturePausedRef.current = false; + setPicturePaused(false); } function forgetRoom(): void { @@ -232,7 +246,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { isCurrentGeneration(generation) && streamRef.current === captured ) { - endSharing("屏幕分享已结束"); + endSharing("已停止分享"); } }, { once: true }, @@ -247,6 +261,91 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { setSwitchingSource(false); } + function commitQuality(id: QualityProfileId): void { + qualityIdRef.current = id; + setQualityId(id); + } + + async function changeQuality(nextId: QualityProfileId): Promise { + if (phase !== "live") { + commitQuality(nextId); + return; + } + + const generation = activeGenerationRef.current; + const activeStream = streamRef.current; + if ( + generation === null || + !activeStream || + !isCurrentGeneration(generation) || + sourceSwitchRef.current || + qualityChangeRef.current + ) { + return; + } + + const token = {}; + qualityChangeRef.current = token; + setChangingQuality(true); + setNotice(null); + const profile = QUALITY_PROFILES[nextId]; + + try { + await applyCaptureProfile(activeStream, profile); + if ( + !isCurrentGeneration(generation) || + qualityChangeRef.current !== token || + streamRef.current !== activeStream + ) { + return; + } + + commitQuality(nextId); + setDetails(captureDetails(activeStream)); + const results = await Promise.all( + [...peersRef.current.values()].map((peer) => peer.updateProfile(profile)), + ); + if ( + isCurrentGeneration(generation) && + qualityChangeRef.current === token + ) { + const failed = results.filter((updated) => !updated).length; + setNotice( + failed > 0 + ? `画质已切换，${failed} 位观众未能应用新的发送参数` + : `画质已切换为 ${profile.label}`, + ); + } + } catch (error) { + if ( + isCurrentGeneration(generation) && + qualityChangeRef.current === token + ) { + setNotice(readableError(error)); + } + } finally { + if (qualityChangeRef.current === token) { + qualityChangeRef.current = null; + setChangingQuality(false); + } + } + } + + function togglePicturePause(): void { + const activeStream = streamRef.current; + if (phase !== "live" || !activeStream) { + return; + } + const nextPaused = !picturePausedRef.current; + if (!setVideoPaused(activeStream, nextPaused)) { + setNotice("当前分享没有可暂停的视频轨道"); + return; + } + picturePausedRef.current = nextPaused; + setPicturePaused(nextPaused); + setNotice(nextPaused ? "画面已暂停，音频不受影响" : "画面已恢复"); + } + function removePeer(peerId: string): void { peersRef.current.get(peerId)?.dispose(); peersRef.current.delete(peerId); @@ -286,7 +385,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { peerId, iceConfig, activeStream, - QUALITY_PROFILES[qualityId], + QUALITY_PROFILES[qualityIdRef.current], { sendSignal: (targetPeerId, payload) => isCurrentGeneration(generation) && signalRef.current === signal @@ -518,7 +617,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { let captured: MediaStream; try { // This must remain the first awaited operation in the button gesture. - captured = await captureDisplay(QUALITY_PROFILES[qualityId]); + captured = await captureDisplay(QUALITY_PROFILES[qualityIdRef.current]); } catch (error) { if (!isCurrentGeneration(generation)) { return; @@ -657,7 +756,8 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { phase !== "live" || generation === null || !isCurrentGeneration(generation) || - sourceSwitchRef.current + sourceSwitchRef.current || + qualityChangeRef.current ) { return; } @@ -670,7 +770,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { let captured: MediaStream; try { // Like initial capture, changing source must begin in this button gesture. - captured = await captureDisplay(QUALITY_PROFILES[qualityId]); + captured = await captureDisplay(QUALITY_PROFILES[qualityIdRef.current]); } catch (error) { if ( isCurrentGeneration(generation) && @@ -699,6 +799,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { } retiringStreamRef.current = previousStream; + setVideoPaused(captured, picturePausedRef.current); streamRef.current = captured; setStream(captured); setDetails(captureDetails(captured)); @@ -713,7 +814,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { peer, replaced: await peer.replaceStream( captured, - QUALITY_PROFILES[qualityId], + QUALITY_PROFILES[qualityIdRef.current], ), }; } catch { @@ -835,7 +936,22 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { + )} + {phase === "live" && ( + - ), - )} - - +
+
+ 画质 +
+ {(Object.keys(QUALITY_PROFILES) as QualityProfileId[]).map( + (id) => ( + + ), + )} +
+
+ {phase !== "live" && ( + )} + {phase !== "live" && ( -
- )} + )} + {room && (
diff --git a/src/client/pages/ViewerPage.tsx b/src/client/pages/ViewerPage.tsx index ee63292..ff22b08 100644 --- a/src/client/pages/ViewerPage.tsx +++ b/src/client/pages/ViewerPage.tsx @@ -534,7 +534,9 @@ export function ViewerPage({ roomId, onAuthorizationRequired }: ViewerPageProps)
- {!relayAvailable && signalStatus === "connected" && ( + {!relayAvailable && + signalStatus === "connected" && + (hostOnline || peerSnapshot) && ( TURN 未配置，严格网络可能无法连接 )} {peerSnapshot?.error && ( diff --git a/src/client/styles.css b/src/client/styles.css index 3375fb1..5cc02c6 100644 --- a/src/client/styles.css +++ b/src/client/styles.css @@ -724,6 +724,15 @@ h3 { grid-template-columns: minmax(0, 1fr) 330px; } + .section-heading { + flex-wrap: wrap; + } + + .broadcast-actions { + width: 100%; + flex-wrap: wrap; + } + .setup-controls { grid-template-columns: minmax(0, 1fr) auto; } @@ -777,18 +786,13 @@ h3 { } @media (max-width: 520px) { - .section-heading, .viewer-title-row { flex-wrap: wrap; } - .broadcast-actions { - width: 100%; - } - .broadcast-actions .button { min-width: 0; - flex: 1; + flex: 1 1 calc(50% - 4px); } .viewer-badges { diff --git a/src/client/webrtc/host-peer.ts b/src/client/webrtc/host-peer.ts index e3b340d..6ab4c12 100644 --- a/src/client/webrtc/host-peer.ts +++ b/src/client/webrtc/host-peer.ts @@ -136,6 +136,28 @@ export class HostPeer { return true; } + async updateProfile(profile: QualityProfile): Promise { + if (this.disposed || !this.videoSender) { + return false; + } + try { + await configureVideoSender(this.videoSender, profile); + } catch (error) { + this.setError(error, "调整画质失败"); + return false; + } + if (this.disposed) { + return false; + } + this.profile = profile; + this.statsAccumulator.bytes = null; + this.statsAccumulator.frames = null; + this.statsAccumulator.timestamp = null; + this.snapshot = { ...this.snapshot, error: null }; + this.emit(); + return true; + } + async acceptSignal(payload: SignalPayload): Promise { if (this.disposed || payload.connectionId !== this.connectionId) { return; diff --git a/tests/host-peer-tracks.test.ts b/tests/host-peer-tracks.test.ts index 50e640e..36720bd 100644 --- a/tests/host-peer-tracks.test.ts +++ b/tests/host-peer-tracks.test.ts @@ -8,7 +8,15 @@ const statsCallbacks: Array<() => void> = []; class FakeSender { failNextReplace = false; - readonly setParameters = vi.fn(async () => undefined); + failNextSetParameters = false; + readonly setParameters = vi.fn( + async (_parameters: RTCRtpSendParameters) => { + if (this.failNextSetParameters) { + this.failNextSetParameters = false; + throw new Error("setParameters failed"); + } + }, + ); readonly replaceTrack = vi.fn(async (track: MediaStreamTrack | null) => { if (this.failNextReplace) { this.failNextReplace = false; @@ -264,7 +272,7 @@ describe("HostPeer source replacement", () => { await expect( peer.replaceStream( createStream(createTrack("video", "next-video"), nextAudio), - QUALITY_PROFILES["720p60"], + QUALITY_PROFILES["1080p30"], ), ).resolves.toBe(true); expect(connection.senders[1]?.track).toBe(nextAudio); @@ -290,6 +298,54 @@ describe("HostPeer source replacement", () => { expect(connection.senders[1]?.track).toBeNull(); }); + it("updates quality parameters without replacing media tracks", async () => { + const video = createTrack("video", "video"); + const audio = createTrack("audio", "audio"); + const peer = createPeer(createStream(video, audio)); + + await expect(peer.start()).resolves.toBe(true); + const connection = FakePeerConnection.latest!; + await expect( + peer.updateProfile(QUALITY_PROFILES["1080p60"]), + ).resolves.toBe(true); + + expect(connection.senders[0]?.replaceTrack).not.toHaveBeenCalled(); + expect(connection.senders[1]?.replaceTrack).not.toHaveBeenCalled(); + expect(connection.senders[0]?.setParameters).toHaveBeenCalledTimes(2); + expect( + connection.senders[0]?.setParameters.mock.calls.at(-1)?.[0], + ).toMatchObject({ + degradationPreference: "balanced", + encodings: [{ maxBitrate: 8_000_000, maxFramerate: 60 }], + }); + }); + + it("can retry the selected quality after a sender update fails", async () => { + const peer = createPeer( + createStream( + createTrack("video", "video"), + createTrack("audio", "audio"), + ), + ); + + await expect(peer.start()).resolves.toBe(true); + const videoSender = FakePeerConnection.latest!.senders[0]!; + videoSender.failNextSetParameters = true; + + await expect( + peer.updateProfile(QUALITY_PROFILES["1080p60"]), + ).resolves.toBe(false); + await expect( + peer.updateProfile(QUALITY_PROFILES["1080p60"]), + ).resolves.toBe(true); + + expect(videoSender.setParameters).toHaveBeenCalledTimes(3); + expect(videoSender.setParameters.mock.calls.at(-1)?.[0]).toMatchObject({ + degradationPreference: "balanced", + encodings: [{ maxBitrate: 8_000_000, maxFramerate: 60 }], + }); + }); + it("rolls the first sender back when the second replacement fails", async () => { const oldVideo = createTrack("video", "old-video"); const oldAudio = createTrack("audio", "old-audio"); diff --git a/tests/quality.test.ts b/tests/quality.test.ts new file mode 100644 index 0000000..9e000e3 --- /dev/null +++ b/tests/quality.test.ts @@ -0,0 +1,90 @@ +import { afterEach, describe, expect, it, vi } from "vitest"; + +import { + applyCaptureProfile, + captureDisplay, + configureVideoSender, + QUALITY_PROFILES, + setVideoPaused, +} from "../src/client/media/quality.ts"; + +function createVideoStream() { + const videoTrack = { + contentHint: "", + enabled: true, + applyConstraints: vi.fn(async () => undefined), + } as unknown as MediaStreamTrack; + const stream = { + getTracks: () => [videoTrack], + getVideoTracks: () => [videoTrack], + getAudioTracks: () => [], + } as unknown as MediaStream; + return { stream, videoTrack }; +} + +afterEach(() => { + vi.unstubAllGlobals(); +}); + +describe("realtime quality controls", () => { + it("bounds initial capture to the selected profile", async () => { + const { stream, videoTrack } = createVideoStream(); + const getDisplayMedia = vi.fn(async () => stream); + vi.stubGlobal("navigator", { mediaDevices: { getDisplayMedia } }); + + await expect(captureDisplay(QUALITY_PROFILES["1080p30"])).resolves.toBe( + stream, + ); + + expect(getDisplayMedia).toHaveBeenCalledWith({ + video: { + width: { ideal: 1920, max: 1920 }, + height: { ideal: 1080, max: 1080 }, + frameRate: { ideal: 30, max: 30 }, + }, + audio: true, + }); + expect(videoTrack.contentHint).toBe("motion"); + }); + + it("changes capture constraints without selecting the source again", async () => { + const { stream, videoTrack } = createVideoStream(); + + await applyCaptureProfile(stream, QUALITY_PROFILES["720p30"]); + + expect(videoTrack.applyConstraints).toHaveBeenCalledWith({ + width: { ideal: 1280, max: 1280 }, + height: { ideal: 720, max: 720 }, + frameRate: { ideal: 30, max: 30 }, + }); + }); + + it("sets the screen-share sender degradation preference", async () => { + const parameters = { + encodings: [], + } as unknown as RTCRtpSendParameters; + const setParameters = vi.fn(async () => undefined); + const sender = { + getParameters: () => parameters, + setParameters, + } as unknown as RTCRtpSender; + + await configureVideoSender(sender, QUALITY_PROFILES["1080p60"]); + + expect(parameters.encodings[0]).toMatchObject({ + maxBitrate: 8_000_000, + maxFramerate: 60, + }); + expect(parameters.degradationPreference).toBe("balanced"); + expect(setParameters).toHaveBeenCalledWith(parameters); + }); + + it("pauses only the video track and can resume it", () => { + const { stream, videoTrack } = createVideoStream(); + + expect(setVideoPaused(stream, true)).toBe(true); + expect(videoTrack.enabled).toBe(false); + expect(setVideoPaused(stream, false)).toBe(true); + expect(videoTrack.enabled).toBe(true); + }); +});