From 9ff7e09888900cbaf1d419cd39d4b1d562483f9d Mon Sep 17 00:00:00 2001 From: AAA <35992542+TNTcraftHIM@users.noreply.github.com> Date: Wed, 19 Aug 2026 02:13:15 +0800 Subject: [PATCH 1/2] feat: add live media quality controls (#14) --- README.md | 2 +- docs/README.md | 1 + docs/project-memory.md | 13 +- docs/research/realtime-quality-adaptation.md | 121 ++++++++++++ docs/research/webrtc-p2p-screen-sharing.md | 4 +- docs/status.md | 70 +++---- docs/方案设计.md | 6 +- docs/需求理解.md | 4 +- src/client/media/quality.ts | 46 +++-- src/client/pages/HostPage.tsx | 198 +++++++++++++++---- src/client/pages/ViewerPage.tsx | 4 +- src/client/styles.css | 16 +- src/client/webrtc/host-peer.ts | 22 +++ tests/host-peer-tracks.test.ts | 60 +++++- tests/quality.test.ts | 90 +++++++++ 15 files changed, 549 insertions(+), 108 deletions(-) create mode 100644 docs/research/realtime-quality-adaptation.md create mode 100644 tests/quality.test.ts diff --git a/README.md b/README.md index 9624cf1..ce52984 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ Screener is a private, low-latency screen-sharing project for one game player and a small group of friends. Its media path is WebRTC P2P-first, with authenticated TURN fallback for network pairs that cannot connect directly. Viewers should be able to join from a desktop or mobile browser without installing the sender application. -The first measurable Web proof of concept is implemented. A host captures a screen, window, or browser tab and shares a numeric room code; friends can open `/r/{code}` directly or enter the code at `/join`, with no separate viewer token. Rooms are random and temporary by default. A password-protected deployment can optionally use SQLite-backed room numbers starting at `1`, reusable links, and a waiting state after sharing stops. Each viewer currently receives an independent WebRTC connection, so direct and TURN-relayed paths can coexist in one room. The default room capacity is eight viewers and deployments may configure 1 through 16, but that admission limit is not a performance promise. +The first measurable Web proof of concept is implemented. A host captures a screen, window, or browser tab and shares a numeric room code; friends can open `/r/{code}` directly or enter the code at `/join`, with no separate viewer token. During a share, the host can change between three ceiling profiles without reopening the source picker: 1080p60 at 8 Mbps, 1080p30 at 5 Mbps, and 720p30 at 3 Mbps. The host can also temporarily pause the picture or switch sources. Rooms are random and temporary by default. A password-protected deployment can optionally use SQLite-backed room numbers starting at `1`, reusable links, and a waiting state after sharing stops. Each viewer currently receives an independent WebRTC connection, so direct and TURN-relayed paths can coexist in one room. The default room capacity is eight viewers and deployments may configure 1 through 16, but that admission limit is not a performance promise. This is not yet a production release. Public TURN/UDP and TURN/TCP relay paths are verified, but real cross-network Screener media, game audio, a sustained 1:8 room, mobile browser lifecycle behavior, and latency targets still require the documented manual test matrix. diff --git a/docs/README.md b/docs/README.md index de3c431..0cc6ff8 100644 --- a/docs/README.md +++ b/docs/README.md @@ -7,6 +7,7 @@ - [Deployment](./deployment.md): minimal Node, HTTPS/WSS, coturn, firewall, secret, optional TURN/TLS, and network-verification procedure. - [Maintenance guide](./maintenance.md): Git workflow, context hygiene, document lifecycle, research policy, and automation. - [P2P WebRTC research](./research/webrtc-p2p-screen-sharing.md): evidence, bandwidth model, browser constraints, reference implementations, and feasibility assessment. +- [Realtime quality adaptation](./research/realtime-quality-adaptation.md): browser degradation policy, live profile changes, offline-encoding boundaries, and measurement gates. - [Agent context governance research](./research/agent-context-governance.md): official Codex, Claude Code, Hermes Agent, and GitHub practices adopted by this repository. - [ADR-0001](./adr/0001-p2p-first-media-topology.md): accepted P2P-first media topology and its consequences. - [ADR-0002](./adr/0002-persistent-protected-rooms.md): optional SQLite persistence, protected sequential room IDs, and room lifecycle. diff --git a/docs/project-memory.md b/docs/project-memory.md index bbaaf75..77fb9af 100644 --- a/docs/project-memory.md +++ b/docs/project-memory.md @@ -1,6 +1,6 @@ # Project Memory -Last updated: 2026-08-18 +Last updated: 2026-08-19 ## Confirmed Intent @@ -9,7 +9,7 @@ Last updated: 2026-08-18 - The normal use case is a small, trusted friend group. Public or large-scale streaming is explicitly out of scope and can be handled by OBS/Twitch-class services. - Viewers should be able to open an invite link in a desktop or mobile browser without installing a dedicated client. - The system should resemble a Photon-style developer experience: a central room/rendezvous service establishes sessions while realtime traffic is carried by clients whenever possible. -- Server bandwidth cost is a primary constraint. The default media path remains P2P-first, with servers used for signaling, STUN, TURN fallback, and observability. A deployment-level SFU mode is acceptable when measured multi-viewer P2P behavior is unusable; it must remain explicit and independently reversible. +- Server bandwidth cost is a primary constraint. The default media path remains P2P-first, with servers responsible for rooms, signaling, STUN, TURN fallback, and observability. Any topology change requires separate measurements and an ADR. - Avoiding TeamSpeak-like partial reachability is a primary requirement: every broadcaster-viewer pair must independently have TURN/UDP and TURN/TCP candidates available when direct ICE cannot connect. TURN/TLS is an optional compatibility enhancement, not a production prerequisite. - A web experience is preferred for convenience, but using a desktop sender is acceptable when it materially improves game capture, audio capture, or hardware encoding. - All project documentation, memory, code, `AGENTS.md`, and `.codex/` configuration must live in this Git repository and remain tracked for cross-device development. @@ -24,22 +24,22 @@ Last updated: 2026-08-18 - Rooms default to one broadcaster and at most eight viewers, with a deployment range of 1 through 16. Treat eight as an admission default rather than a validated performance claim; measure publisher upload, encoder load, latency, and stability before describing a supported 1:8 envelope. - Use HTTPS/WSS signaling, trickle ICE, STUN, and authenticated coturn candidates. Prefer direct UDP, then relay UDP, with TURN/TCP as the required non-UDP fallback. Optional TURN/TLS uses standard TCP 5349 by default; TCP 443 is reserved for deployments with a dedicated public IP or validated L4/SNI routing. - Allow mixed connectivity in one room: direct viewers stay direct while only incompatible network pairs consume TURN bandwidth. -- Treat 1080p60 as a best-effort quality profile, not a universal guarantee. Provide 720p60 and 720p30 fallbacks. +- Treat 1080p60 at 8 Mbps as a best-effort ceiling, not a universal guarantee. Keep three simple manual profiles: 1080p60 at 8 Mbps, 1080p30 at 5 Mbps, and 720p30 at 3 Mbps. Live profile changes do not reopen the source picker. Keep `contentHint = "motion"` and explicit `balanced` degradation, but do not claim resolution-first behavior; actual frame-rate and resolution tradeoffs require browser, hardware, codec, and network measurements. - Do not add custom scene detection or dynamic-FPS control until WebRTC statistics and host resource measurements show that browser capture, encoding, and congestion behavior leave a material problem. - Leave codec order at the browser default in the first PoC and record the negotiated codec, encoder implementation, and power efficiency. Prefer H.264 only after target-machine measurements show that it is the hardware-efficient path; retain VP8 compatibility. - Add an Electron or native Windows sender only after browser measurements identify capture, application-audio, or encode bottlenecks. -- Keep P2P as the default and add only an explicit deployment-level `p2p|sfu` choice. Multi-viewer use has already shown severe user-observed P2P degradation, consistent with one sender and encoder per viewer. The first SFU experiment should use a single node, one published layer, and no automatic switching, hybrid path, Redis, or peer forwarding tree. +- Keep one independent publisher-to-viewer connection in the Web PoC until a separately measured and reversible topology decision is accepted. - Keep room policy deployment-driven and small. Public and password-only deployments use random temporary rooms. A deployment that configures both the whole-site password and a SQLite path gets sequential, non-expiring protected rooms; stopping a share leaves viewers waiting and does not destroy the room. ## Current Implementation - The repository contains a single npm package using Node.js 24, React, TypeScript, Vite, native WebRTC, `ws`, Zod, Vitest, and a separate coturn deployment. The server defaults to an all-interface listener for LAN development and containers; the bare-metal reverse-proxy baseline explicitly binds loopback. It provides a process-only `/healthz` endpoint and requires STUN plus explicit TURN/UDP and TURN/TCP URLs before production startup. TURN/TLS is accepted but optional; configuration checks do not establish public-network reachability. -- The Web PoC implements capture-before-room creation, live source replacement without renegotiating healthy peers, one independent peer connection per viewer, stable signaling reconnect identities that also work for LAN viewers on HTTP, explicit ICE restart or peer rebuild, host session generation isolation, viewer connection-generation guards for asynchronous signaling and stats, short-lived TURN credentials, three manual quality profiles, and local WebRTC statistics. +- The Web PoC implements capture-before-room creation, live source and profile changes without renegotiating healthy peers, three manual quality profiles with an explicit `balanced` sender preference, temporary picture pause with audio unaffected, one independent peer connection per viewer, stable signaling reconnect identities that also work for LAN viewers on HTTP, explicit ICE restart or peer rebuild, host session generation isolation, viewer connection-generation guards for asynchronous signaling and stats, short-lived TURN credentials, clearer waiting/TURN states, and local WebRTC statistics. - Access is deliberately small: `ACCESS_PASSWORD` is optional, site-wide, and accepts 1 through 128 visible ASCII characters when non-empty. When configured, host and viewer routes first establish a 12-hour stateless HMAC HttpOnly `SameSite=Strict` cookie; room creation and WebSocket upgrade accept that cookie and no direct room-creation Bearer bypass. There are no accounts, JWTs, server-side access-session maps, or logout flow. - ADR-0002 accepts an optional `ROOM_DATABASE_PATH` only alongside `ACCESS_PASSWORD`. In that mode, built-in `node:sqlite` stores only an auto-incremented room ID and host-token digest; links do not expire and stopping sharing leaves the room waiting. Without the path, rooms remain random and temporary. Public mode can never use sequential persistent rooms. Commit `2f66770f8e90` is deployed with `/var/lib/screener/rooms.sqlite`; the state directory and database permissions are verified, and the empty database preserves ID `1` for the first real browser room. - `/r/{code}` carries no viewer token or fragment, `/join` accepts only the numeric code, and the 256-bit host token stays internal to host authentication. In public mode the random code is the sole viewing capability and is not a strong privacy guarantee, so private Internet deployments should configure `ACCESS_PASSWORD`. - The Web control plane is deployed at `https://share.bonfire.icu` behind nginx with Node.js 24.19.0, and `turn.bonfire.icu` runs authenticated coturn 4.17.2 on standard UDP/TCP 3478. HTTPS, WSS, room authentication, certificate renewal, public STUN, authenticated TURN/UDP and TURN/TCP allocations, and relay-only bidirectional data paths are verified. TURN/TLS is intentionally not enabled. -- Automated checks pass on current `main` with 102 Vitest tests and both production builds. Same-machine synthetic-media Chromium recovery and public relay-only DataChannel evidence remains from the earlier baseline. Production storage startup and permissions are verified; a real persistent-room stop/reuse/restart cycle, real screen/game audio, heterogeneous media sessions, mobile lifecycle handling, and latency or quality targets remain unverified. +- Automated checks pass 108 Vitest tests and both production builds; production remains at the earlier deployed revision. Same-machine synthetic-media Chromium recovery and public relay-only DataChannel evidence remains from that baseline. A real live quality/pause cycle, persistent-room stop/reuse/restart cycle, real screen/game audio, heterogeneous media sessions, mobile lifecycle handling, and latency or quality targets remain unverified. - No infrastructure blocker remains for the current staging deployment. The immediate control-plane milestone and the separate real-device/media matrix are bounded in `docs/status.md`; neither justifies a native sender yet. ## Provisional Quality Targets @@ -65,6 +65,7 @@ Last updated: 2026-08-18 - First PoC technical design: `docs/方案设计.md` - Minimal production-shaped deployment: `docs/deployment.md` - Research and feasibility: `docs/research/webrtc-p2p-screen-sharing.md` +- Realtime quality policy: `docs/research/realtime-quality-adaptation.md` - Topology decision: `docs/adr/0001-p2p-first-media-topology.md` - Persistent protected-room decision: `docs/adr/0002-persistent-protected-rooms.md` - Current phase and next step: `docs/status.md` diff --git a/docs/research/realtime-quality-adaptation.md b/docs/research/realtime-quality-adaptation.md new file mode 100644 index 0000000..06d0cd4 --- /dev/null +++ b/docs/research/realtime-quality-adaptation.md @@ -0,0 +1,121 @@ +# Realtime Screen-Share Quality Adaptation + +- Research date: 2026-08-19 +- Scope: realtime game screen sharing in the browser +- Status: implementation input; real-device quality remains unverified + +## Finding + +Screener should not optimize a single quality dimension at the expense of the +others. The previous combination of `contentHint = "motion"` and an explicit +`degradationPreference = "maintain-framerate"` told WebRTC to preserve frame +rate primarily by reducing resolution. That is appropriate for some fast +motion, but it can make game UI, maps, subtitles, and text unreadable. + +The smallest correction is to retain the game-oriented `motion` hint while +explicitly selecting `balanced` degradation. The WebRTC API allows the user +agent to balance frame-rate and resolution degradation; it does not prescribe +the algorithm or guarantee a clarity-first result. Congestion control remains +authoritative, and Chromium, Firefox, and Safari behavior must be measured. + +The inspected Chromium/libwebrtc source chain makes a screen-only shortcut +especially unsafe to assume for Screener. The JavaScript `motion` hint reaches +`VideoRtpSender` as `kFluid`, which explicitly clears `options.is_screencast`. +`WebRtcVideoSendStream` then configures the encoder as `kRealtimeVideo`, while +`VideoStreamEncoder` only reinterprets `BALANCED` as `MAINTAIN_RESOLUTION` when +its encoder is marked as screen share. The explicit `balanced` preference still +wins the earlier preference-selection branch, but the later screen-only remap +does not follow from this project path. Treat the effective tradeoff as an +implementation detail to verify, not a product guarantee. + +An older libwebrtc experiment could detect animated screen content under +`balanced` and cap it to 1280x720. Commit `1d7d0e6e2c50` deleted that experiment +on 2024-05-22; the commit says it had already been disabled for several years +and was not maintained. Current Chromium behavior must therefore not be +described as automatically detecting game motion and forcing a 720p cap. + +## Why Offline Encoding Presets Do Not Transfer + +[Maruko Toolbox](https://github.com/wzxjohn/marukotoolbox) is an Apache-2.0 +GUI around offline video-processing tools. Its x264-style CRF, slow presets, +lookahead, and multi-pass workflows trade encoding time and buffering for file +size and quality. Interactive WebRTC cannot buffer or revisit future frames in +that way. The transferable principle is to treat bitrate, image detail, frame +rate, and encode cost as a joint tradeoff; its concrete offline presets are not +a WebRTC configuration baseline. +The GUI repository is Apache-2.0, while x264 itself uses GPL-2.0 or commercial +licensing; no implementation or preset is copied into Screener. + +Discord's published Go Live design reaches the same broad conclusion for +realtime game streaming: bandwidth estimates, frame delivery, image quality, +latency, CPU, and memory conflict and must be balanced. Its native encoder +tuning and hardware integration are not available to a browser-only sender and +must not be presented as settings this project already has. + +## Current Policy + +The three user-visible profiles remain ceilings rather than promised rates: + +| Profile | Capture ceiling | Sender bitrate ceiling | +| --- | --- | ---: | +| 1080p60 | 1920x1080 at 60 fps | 8 Mbps | +| 1080p30 | 1920x1080 at 30 fps | 5 Mbps | +| 720p30 | 1280x720 at 30 fps | 3 Mbps | + +The middle profile is an explicit manual 1080p30 ceiling: choosing it trades a +60 fps ceiling for a 1080p capture bound. LiveKit currently uses the same +1080p30 at 5 Mbps screen-share preset, but neither preset guarantees the emitted +resolution, frame rate, or bitrate. + +- Initial capture supplies `ideal` and `max` bounds for resolution and frame + rate; the actual result is read back from `MediaStreamTrack.getSettings()`. +- A live profile change uses `track.applyConstraints()` and updates every + current sender with `RTCRtpSender.setParameters()`. It does not reopen the + source picker or renegotiate healthy peer connections. +- The video track keeps `contentHint = "motion"`, while each sender explicitly + uses `degradationPreference = "balanced"`. This is a user-agent preference, + not a promise to preserve either resolution or frame rate. +- `maxBitrate` and `maxFramerate` are ceilings. They are neither minimums nor + target guarantees, and the project does not use SDP bitrate hacks. +- Pausing the picture disables the existing video track, producing black video + without closing the room or media connection. Audio remains enabled. + +## Deliberate Non-Goals + +- No canvas pixel-difference detector, machine-learned rate controller, or + periodic profile switching. +- No copied x264 CRF/preset recipe in the browser path. +- No forced codec order until target hardware measurements identify the actual + power-efficient encoder. +- No promise that all browsers honor `applyConstraints` or degradation + preference identically. + +## Verification Gate + +Use the same static UI scene and deterministic high-motion game scene at +720p30, 1080p30, and 1080p60. Record actual capture and outbound dimensions/fps, +bitrate, `qualityLimitationReason`, codec, encoder implementation, encode time, +dropped frames, RTT, packet loss, and host CPU/GPU utilization. A live profile +change must preserve peer connection IDs, avoid a second source prompt, and +visibly converge to the requested bounds. + +Compare image readability and motion continuity instead of declaring success +from FPS alone. If reproducible evidence later shows that `balanced` still +oscillates or makes the wrong tradeoff on supported machines, revise the fixed +profiles before adding an application-level adaptation controller. + +## Primary Sources + +- [W3C MediaStreamTrack Content Hints](https://www.w3.org/TR/mst-content-hint/) +- [W3C Screen Capture](https://www.w3.org/TR/screen-capture/) +- [W3C WebRTC](https://www.w3.org/TR/webrtc/) +- [Chromium `motion` to libwebrtc `kFluid` bridge](https://chromium.googlesource.com/chromium/src/third_party/+/refs/heads/main/blink/renderer/modules/peerconnection/media_stream_video_webrtc_sink.cc) +- [libwebrtc `motion`/`kFluid` sender classification](https://webrtc.googlesource.com/src/+/3b1eab8a69cb5078befb021c5492d3f204a7d6a2/pc/rtp_sender.cc) +- [libwebrtc encoder content-type selection](https://webrtc.googlesource.com/src/+/9caef2a8b88f389af10cee841732c42a98d3d45d/media/engine/webrtc_video_engine.cc) +- [libwebrtc conditional screen-share degradation mapping](https://webrtc.googlesource.com/src/+/refs/heads/main/video/video_stream_encoder.cc) +- [libwebrtc removal of the automatic animation-detection experiment, 2024-05-22](https://webrtc.googlesource.com/src/+/1d7d0e6e2c5002815853be251ce43fe88779ac85) +- [LiveKit screen-share presets](https://github.com/livekit/client-sdk-js/blob/main/src/room/track/options.ts) +- [LiveKit degradation defaults](https://github.com/livekit/client-sdk-js/blob/main/src/room/participant/publishUtils.ts) +- [Jitsi desktop degradation preference](https://github.com/jitsi/lib-jitsi-meet/blob/master/modules/RTC/TraceablePeerConnection.ts) +- [Discord Go Live architecture](https://discord.com/blog/how-it-all-goes-live-an-overview-of-discords-streaming-technology) +- [Discord encoder-quality case study](https://discord.com/blog/from-blocky-to-brilliant-improving-video-quality-on-discord-go-live-on-amd-gpus) diff --git a/docs/research/webrtc-p2p-screen-sharing.md b/docs/research/webrtc-p2p-screen-sharing.md index 906dbd3..018e837 100644 --- a/docs/research/webrtc-p2p-screen-sharing.md +++ b/docs/research/webrtc-p2p-screen-sharing.md @@ -152,11 +152,11 @@ const parameters = sender.getParameters(); parameters.encodings ??= [{}]; parameters.encodings[0].maxFramerate = 60; parameters.encodings[0].maxBitrate = selectedProfileBitrate; -parameters.degradationPreference = "maintain-framerate"; +parameters.degradationPreference = "balanced"; await sender.setParameters(parameters); ``` -这些参数是偏好或上限，不能绕过浏览器拥塞控制，也不能保证目标码率。Screen Capture 和 WebRTC 规范允许浏览器决定实际输出帧率、编码与拥塞行为；`contentHint = "motion"` 也只是内容提示。静态画面通常能降低编码数据量，但规范不保证所有浏览器主动降低捕获频率或 GPU 开销。 +这些参数是偏好或上限，不能绕过浏览器拥塞控制，也不能保证目标码率。`balanced` 允许浏览器在分辨率和帧率之间权衡，但不规定具体算法；本项目的 `contentHint = "motion"` 还会影响 Chromium/libwebrtc 对来源的内部分类，因此不能套用其仅限 screen encoder 的 `maintain-resolution` 重解释。当前源码链与测量要求见 [Realtime quality adaptation](./realtime-quality-adaptation.md)。静态画面通常能降低编码数据量，但规范不保证所有浏览器主动降低捕获频率或 GPU 开销。 因此首版不实现画面差分检测、周期性 `applyConstraints()` 或自定义动态 FPS 状态机。先对静态桌面和高动态游戏分别记录 `framesEncoded`、发送码率、`totalEncodeTime / framesEncoded`、`qualityLimitationReason` 及主机 CPU/GPU 占用；只有测量显示浏览器行为留下显著问题时，再设计最小的控制策略。这避免用额外竞态、计时器和画质跳变解决一个可能已由捕获器、编码器和 WebRTC 拥塞控制处理的问题。 diff --git a/docs/status.md b/docs/status.md index 84cf1cf..2b825ac 100644 --- a/docs/status.md +++ b/docs/status.md @@ -1,6 +1,6 @@ # Current Status -Last updated: 2026-08-18 +Last updated: 2026-08-19 ## Phase @@ -8,63 +8,53 @@ The WebRTC proof of concept is deployed at `https://share.bonfire.icu`. Commit `2f66770f8e90` adds protected SQLite persistence, sequential room IDs, reusable links, and a waiting state after sharing stops. The production database is initialized and empty, so the first real persistent room can receive ID `1`. -Real capture, game audio, mobile lifecycle, and performance targets remain -unverified. Increasing the P2P viewer count has already caused severe user- -observed degradation, so an explicitly configured SFU mode is the next media -experiment; P2P remains the default topology. + +The Web PoC now includes live quality changes, an explicit `balanced` sender +degradation preference, +temporary picture pause with audio unaffected, and clearer stopped/waiting and +TURN-warning states. These controls are not deployed yet. Real capture, game +audio, mobile lifecycle, sustained multi-viewer behavior, and performance +targets remain unverified. ## Established Baseline - Runtime: Node.js 24, React, TypeScript, Vite, native browser WebRTC, `ws`, Zod, and a separate coturn deployment. - Product scope: private game sharing for one broadcaster and a small friend group. Rooms default to eight viewers and accept a configured limit from 1 through 16; this is an admission limit, not a verified performance envelope. -- Client behavior: screen capture precedes room creation; live source changes preserve healthy peer connections; `/r/{code}` invitations and `/join` use a numeric room code with no viewer token or URL fragment; every viewer has an independent `RTCPeerConnection`. -- Control plane contract: optional whole-site `ACCESS_PASSWORD` with 1 through 128 visible ASCII characters, a 12-hour stateless HMAC HttpOnly `SameSite=Strict` cookie, an internal host token, role-bound signaling, stable reconnect identities, short-lived coturn credentials, and Origin/payload checks. There are no accounts, JWTs, server-side access-session maps, or logout flow. -- Room policies: no `ROOM_DATABASE_PATH` means random temporary rooms governed by `ROOM_TTL_SECONDS`; configuring it together with `ACCESS_PASSWORD` enables protected rooms whose numeric IDs increment from `1` and whose links do not expire. A database path without the whole-site password is invalid. Stopping sharing leaves the room available and viewers waiting. SQLite stores only room ID and host-token digest. -- Media topology: direct ICE is preferred per viewer, with authenticated TURN/UDP and TURN/TCP required in production. TURN/TLS is an optional deployment capability, using standard TCP 5349 by default; mixed direct and relay paths are supported by design. -- Diagnostics: per-peer path, candidate types, ICE protocol, local TURN protocol, RTT, bitrate, frame rate, dimensions, loss, jitter, codec, and quality-limitation fields are read locally from WebRTC stats. -- Large public broadcasts are out of scope and should use OBS/Twitch-class services. +- Client behavior: capture precedes room creation; live source and quality changes preserve healthy peer connections; picture pause keeps audio and connections active; `/r/{code}` and `/join` use a numeric room code; every viewer has an independent `RTCPeerConnection`. +- Access control: an optional whole-site `ACCESS_PASSWORD` issues a 12-hour stateless HMAC HttpOnly `SameSite=Strict` cookie. Room creation and WebSocket upgrade accept that cookie with no Bearer bypass; there are no accounts, JWTs, server-side access sessions, or logout flow. +- Room policy: protected deployments may configure SQLite-backed sequential rooms whose links survive stopped sharing and process restarts. Other rooms remain random and temporary. SQLite stores only room ID and host-token digest. +- Media path: direct ICE is preferred per viewer, with authenticated TURN/UDP and TURN/TCP required in production. TURN/TLS remains optional, and direct and relay paths may coexist in one room. +- Diagnostics: peer path, candidates, ICE protocol, local TURN protocol, RTT, bitrate, frame rate, dimensions, loss, jitter, codec, encode time, and quality limitations are read from local WebRTC stats. +- Large public broadcasts remain out of scope and should use OBS/Twitch-class services. ## Verification -- On current `main`, `npm run check` passes type checking, - 102 Vitest tests, the client production build, and the server production build. - -The remaining browser and network evidence predates the persistent-room rollout; -it does not verify a real browser stop-and-republish cycle with room ID `1`. - -- Headless Chromium 151 created real local peer connections from animated canvas streams. The viewer reached `connected`, received a live video track, and reported a direct UDP path. -- Browser fault injection recovered after dropping the first offer, dropping the first answer, and failing the first `createOffer`. A viewer-only signaling reconnect preserved the existing healthy media peer; replacing that viewer tab rebuilt media, released the old peer, and did not enter a reconnect loop. -- Delayed ViewerPeer answer, candidate flush, ICE event, and stats results are discarded after a connection generation is replaced, so an old peer cannot signal through or overwrite the new peer snapshot. -- Injecting a live picker-cancellation result preserved the old stream. Replacing synthetic video, adding synthetic audio, and removing it all kept the same connected host/viewer peer objects and did not create another offer; retired capture tracks stopped after each successful change. -- Chrome 151 at 390 px completed the protected-site gate, login, host, and manual room-code join flow without horizontal overflow. The unauthenticated gate shows no application header or sharing controls. -- Production configuration fails closed when required HTTPS, STUN, TURN, or TURN-secret constraints are missing. -- Production startup rejects malformed ICE URLs and requires STUN plus explicit TURN/UDP and TURN/TCP entries. Optional TURN/TLS entries may use standard TCP 5349 or, when separately routed, TCP 443. Configuration validation remains distinct from the runtime relay checks below. -- The application defaults to `LISTEN_HOST=0.0.0.0` for LAN development and containers, while the bare-metal reverse-proxy deployment explicitly uses loopback. It exposes a process-only `GET /healthz` liveness response. -- Viewer session identities use `crypto.getRandomValues()` rather than the secure-context-only `crypto.randomUUID()`, so a phone can initialize the viewer over trusted LAN HTTP while the host keeps screen capture on `localhost`. -- The staging host runs Debian 12, nginx, Node.js 24.19.0, and source-built coturn 4.17.2. Screener binds loopback behind nginx; authenticated STUN and TURN/UDP+TCP bind `turn.bonfire.icu:3478`. TURN/TLS remains intentionally disabled. -- Commit `2f66770f8e90` is deployed. Production verification covered rejection of the previous password, login with the current password, the `Secure`/`HttpOnly`/`SameSite=Strict` cookie, the initialized empty SQLite schema, service state directory mode `0700`, database mode `0600`, and a clean service restart with zero automatic restarts. The prior room and WebSocket authentication checks remain applicable. The existing blog continued returning HTTP 200. -- Public STUN Binding succeeds over both UDP and TCP 3478. Chromium 151 obtained authenticated relay candidates through TURN/UDP and TURN/TCP, then two relay-only peer connections completed a bidirectional DataChannel ping/pong over each transport. Simultaneous guest conntrack samples observed both 3478 control traffic and relay-range UDP traffic, independently confirming the public TURN path through the cloud and host firewalls. -- The existing TeamSpeak files, services, timers, and ports were not changed. +- `npm run check` passes type checking, 108 Vitest tests, and both production builds. +- Headless Chromium 151 established real local peer connections from animated canvas streams, delivered a live video track over direct UDP, and recovered from dropped offer/answer and initial `createOffer` failures. +- Host and viewer generation guards discard delayed signaling, candidate, and stats work after replacement. Synthetic source changes added and removed audio without another offer, preserved healthy peer objects, and stopped retired tracks. +- Unit coverage verifies live capture constraints, balanced sender parameters, video-only pause, sender-update failure and retry, source replacement rollback, and audio `null -> track -> null` changes. +- Headless Chrome layout checks at actual inner widths 500, 781, and 820 px keep all live controls inside the host area, including the forced-relay badge. Chrome clamped the requested 390 px window to 500 px, so a true 390 px browser viewport remains unverified. +- Production configuration fails closed without HTTPS, STUN, TURN credentials, explicit TURN/UDP, or explicit TURN/TCP. Public STUN and authenticated relay-only DataChannel tests passed over TURN/UDP and TURN/TCP. +- Staging runs Debian 12, nginx, Node.js 24.19.0, and coturn 4.17.2. Screener binds loopback behind nginx; the protected SQLite directory and database permissions survive a clean service restart. TURN/TLS is intentionally disabled. ## Next Milestone -Preserve the first real room for the normal browser workflow, then verify room ID -`1`, stop-and-republish waiting behavior, link reuse, and recovery after a service -restart. In a separate PR and ADR, add a deployment-level `p2p|sfu` media mode -using a single-node SFU; do not add automatic switching, hybrid sending, Redis, -or multi-node infrastructure before measurements justify them. +Merge and deploy the live controls, then verify a real quality/pause cycle, room +ID `1`, stop-and-republish behavior, reusable links, and recovery after a service +restart. -Execute and record the manual browser/network matrix for both applicable modes: +Execute and record the manual browser/network matrix: - real screen/window capture and available game or system audio on Windows Chrome and Edge; -- one broadcaster with 1, 3, 5, and 8 heterogeneous viewers, recording publisher upload and encode load plus server ingress and egress to locate each sustainable envelope; -- same-LAN direct, cross-network direct, a forced-relay Screener media session, mixed direct/relay, and a UDP-blocked TURN/TCP path; test TURN/TLS separately only when deployed; +- one broadcaster with 1, 3, 5, and 8 heterogeneous viewers, recording publisher upload, encode load, and server traffic; +- same-LAN direct, cross-network direct, forced relay, mixed direct/relay, and UDP-blocked TURN/TCP paths; - Android Chrome and iOS Safari playback, orientation, backgrounding, and network handoff; -- actual codec implementation, encode load, publisher upload, bitrate, frame rate, first picture, and glass-to-glass latency. Custom dynamic-FPS logic remains intentionally unimplemented until these measurements show a gap in browser capture, encoding, and congestion behavior. +- live 1080p60 at 8 Mbps, 1080p30 at 5 Mbps, and 720p30 at 3 Mbps changes without a second source prompt or peer rebuild, plus picture pause/resume while audio continues; +- codec implementation, encode load, bitrate, frame rate, first picture, and glass-to-glass latency. ## Current Blocker -- No infrastructure blocker remains for the current deployment. Persistent storage and systemd filesystem permissions are active; the first real browser room lifecycle and the separate P2P/SFU media matrix remain unverified. +- No infrastructure blocker remains. The live-control browser cycle, first real persistent-room lifecycle, and real-device media matrix remain unverified. ## Blocking Decisions diff --git a/docs/方案设计.md b/docs/方案设计.md index 45242e2..ba60174 100644 --- a/docs/方案设计.md +++ b/docs/方案设计.md @@ -130,10 +130,10 @@ Zod 在服务边界拒绝未知类型、超长字符串、非法 SDP/candidate | 档位 | 约束目标 | 初始码率上限 | | --- | --- | ---: | | 1080p60 | 1920x1080、60 fps | 8 Mbps/peer | -| 720p60 | 1280x720、60 fps | 5 Mbps/peer | +| 1080p30 | 1920x1080、30 fps | 5 Mbps/peer | | 720p30 | 1280x720、30 fps | 3 Mbps/peer | -浏览器可返回更低设置，界面以 `track.getSettings()` 显示实际结果。视频 track 设置 `contentHint = "motion"`；sender 尝试设置 `maxBitrate`、`maxFramerate` 与 `degradationPreference = "maintain-framerate"`，不支持时只记录警告。 +浏览器可返回更低设置，界面以 `track.getSettings()` 显示实际结果。视频 track 保留 `contentHint = "motion"`；sender 尝试设置 `maxBitrate`、`maxFramerate` 与 `degradationPreference = "balanced"`。`balanced` 只允许浏览器在帧率和分辨率之间权衡，不保证清晰度优先；具体 Chromium、Firefox 与 Safari 行为仍需实测。不支持 sender 参数时只记录警告。 PoC 不强制 codec，也不做 SDP munging。先通过 stats 记录实际协商 codec、`encoderImplementation` 和 `powerEfficientEncoder`，再依据目标机器测量决定是否优先 H.264。这样避免静态 codec 偏好意外触发软件编码。 @@ -141,6 +141,8 @@ PoC 不强制 codec，也不做 SDP munging。先通过 stats 记录实际协商 直播中切换来源仍由明确的按钮手势重新调用 `getDisplayMedia()`，不改变房间、host generation 或媒体拓扑。选源取消或失败时继续发送旧流；成功后为每个现有 sender 调用 `replaceTrack()`，健康连接不重新协商。初始 offer 预留 send-only audio transceiver，使音频可以在 `null -> track -> null` 间切换。若某个 peer 的替换或回滚失败，只重建该 peer；新流提交后才停止旧流，并以当前 stream 身份保护旧 track 的迟到 `ended` 事件。统计码率累加器在换源后重置，避免把不同 track 的计数器混算。 +直播中切换质量档位不重新打开选源器：对当前视频 track 调用 `applyConstraints()`，再更新所有现有 sender 的参数；新加入或重建的 sender 使用当前档位。失败时保留当前分享并给出状态，不以重建整个房间补偿。临时暂停只禁用当前视频 track，使远端收到黑画面；音频、房间和连接保持不变，恢复时重新启用同一 track。 + `frameRate` 约束和 sender 的 `maxFramerate` 都是目标或上限；浏览器仍控制实际捕获、重复帧处理、编码和拥塞降级。静态画面可能由浏览器减少实际编码数据，但标准没有保证每个实现以同一种方式节省采集或 GPU 开销。本 PoC 不添加画面差分、定时改约束或自定义动态 FPS 状态机，先用 `framesEncoded`、实际发送码率、`totalEncodeTime`、质量受限原因和主机资源占用测量真实问题。 自动全房间降档同样需要可信的聚合策略，放在测量后的下一阶段；本 PoC 提供手动档位和明确的受限原因，不伪装成已经解决自适应问题。 diff --git a/docs/需求理解.md b/docs/需求理解.md index 8e4f2f3..bc287f0 100644 --- a/docs/需求理解.md +++ b/docs/需求理解.md @@ -14,12 +14,12 @@ - [ ] 简单房间：分享者获得纯数字房间码和 `/r/{code}` 链接；观看者也可在 `/join` 只输入房间码加入，链接不携带额外观看 token 或 fragment。启用持久受保护房间时，房间码从 `1` 开始自增；其他模式继续使用随机临时房间码防止枚举。 - [ ] 免安装观看：桌面和手机观看者可直接通过浏览器链接进入、授权并播放，不依赖与分享端相同的桌面客户端。 -- [ ] 屏幕采集：分享者可主动选择显示器或应用窗口，开始、停止和切换分享状态。 +- [ ] 屏幕采集：分享者可主动选择显示器或应用窗口，开始、停止、临时暂停画面和切换分享来源。 - [ ] 游戏声音：Windows Chrome/Edge 目标环境可请求系统音频；没有返回音轨时必须明确提示，而不是静默无声。 - [ ] 一对多观看：一名分享者可向配置允许的多名观看者发送画面，每名观看者拥有独立的 WebRTC 连接与状态；默认上限为 8，可配置范围为 1 至 16。 - [ ] P2P 优先：ICE 必须先尝试直接 UDP 路径，服务器不得默认中转可直连的媒体。 - [ ] NAT 兜底：每名观看者独立进行 ICE；生产环境必须提供 STUN、TURN/UDP 与 TURN/TCP candidates，通过优先级偏好直连和 UDP，同时允许无法使用 UDP 的网络选择 TCP，不能因为其网络与其他观看者不同而静默失败。 -- [ ] 质量档位：至少提供 1080p60、720p60、720p30 三档目标，并依赖 WebRTC 拥塞控制及明确的手动档位避免持续排队；自定义动态 FPS 策略必须在统计证明有必要后再实现。 +- [ ] 质量档位：提供 1080p60（8 Mbps）、1080p30（5 Mbps）、720p30（3 Mbps）三档上限；分享期间可直接换档而不重新选择来源或重建健康连接。保留 `contentHint = "motion"` 并显式设置 `degradationPreference = "balanced"`；该偏好允许浏览器在帧率和分辨率之间权衡，不保证清晰度优先，具体行为必须实测。自定义动态 FPS 或画面检测策略必须在统计证明有必要后再实现。 - [ ] 断线恢复：临时网络变化时尝试 ICE restart 和重连，并向用户显示当前连接状态。 - [ ] 可观测性：采集候选类型、RTT、丢包、抖动、实际码率、帧率、编码耗时和质量受限原因。 - [ ] 访问控制：部署者可配置一个全站 `ACCESS_PASSWORD`；为空时网站公开，非空值必须为 1 至 128 个可见 ASCII 字符。配置时分享者和观看者都必须先登录。登录成功签发 12 小时、无服务端会话状态的 HttpOnly、`SameSite=Strict` cookie，建房 HTTP 和 WebSocket 都只使用该 cookie，不提供建房 Bearer 旁路；不引入账号、JWT、服务端 session Map 或 logout。 diff --git a/src/client/media/quality.ts b/src/client/media/quality.ts index 00c759d..2a05058 100644 --- a/src/client/media/quality.ts +++ b/src/client/media/quality.ts @@ -6,11 +6,11 @@ export const QUALITY_PROFILES = { frameRate: 60, maxBitrate: 8_000_000, }, - "720p60": { - label: "720p 60", - width: 1280, - height: 720, - frameRate: 60, + "1080p30": { + label: "1080p 30", + width: 1920, + height: 1080, + frameRate: 30, maxBitrate: 5_000_000, }, "720p30": { @@ -25,6 +25,14 @@ export const QUALITY_PROFILES = { export type QualityProfileId = keyof typeof QUALITY_PROFILES; export type QualityProfile = (typeof QUALITY_PROFILES)[QualityProfileId]; +function captureConstraints(profile: QualityProfile): MediaTrackConstraints { + return { + width: { ideal: profile.width, max: profile.width }, + height: { ideal: profile.height, max: profile.height }, + frameRate: { ideal: profile.frameRate, max: profile.frameRate }, + }; +} + export async function captureDisplay( profile: QualityProfile, ): Promise { @@ -33,11 +41,7 @@ export async function captureDisplay( } const stream = await navigator.mediaDevices.getDisplayMedia({ - video: { - width: { ideal: profile.width }, - height: { ideal: profile.height }, - frameRate: { ideal: profile.frameRate, max: profile.frameRate }, - }, + video: captureConstraints(profile), audio: true, }); @@ -50,6 +54,26 @@ export async function captureDisplay( return stream; } +export async function applyCaptureProfile( + stream: MediaStream, + profile: QualityProfile, +): Promise { + const videoTrack = stream.getVideoTracks()[0]; + if (!videoTrack) { + throw new Error("共享流缺少视频轨道"); + } + await videoTrack.applyConstraints(captureConstraints(profile)); +} + +export function setVideoPaused(stream: MediaStream, paused: boolean): boolean { + const videoTrack = stream.getVideoTracks()[0]; + if (!videoTrack) { + return false; + } + videoTrack.enabled = !paused; + return true; +} + export async function configureVideoSender( sender: RTCRtpSender, profile: QualityProfile, @@ -60,6 +84,6 @@ export async function configureVideoSender( } parameters.encodings[0].maxBitrate = profile.maxBitrate; parameters.encodings[0].maxFramerate = profile.frameRate; - parameters.degradationPreference = "maintain-framerate"; + parameters.degradationPreference = "balanced"; await sender.setParameters(parameters); } diff --git a/src/client/pages/HostPage.tsx b/src/client/pages/HostPage.tsx index 81555c8..b367db0 100644 --- a/src/client/pages/HostPage.tsx +++ b/src/client/pages/HostPage.tsx @@ -3,6 +3,8 @@ import { Copy, Hash, MonitorUp, + Pause, + Play, RefreshCw, Square, Users, @@ -31,8 +33,10 @@ import { } from "../lib/session"; import { SignalingClient } from "../lib/signaling"; import { + applyCaptureProfile, captureDisplay, QUALITY_PROFILES, + setVideoPaused, type QualityProfileId, } from "../media/quality"; import type { @@ -116,6 +120,8 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { const [notice, setNotice] = useState(null); const [copied, setCopied] = useState(false); const [switchingSource, setSwitchingSource] = useState(false); + const [changingQuality, setChangingQuality] = useState(false); + const [picturePaused, setPicturePaused] = useState(false); const videoRef = useRef(null); const streamRef = useRef(null); @@ -125,6 +131,9 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { const generationRef = useRef(0); const activeGenerationRef = useRef(null); const sourceSwitchRef = useRef(null); + const qualityChangeRef = useRef(null); + const qualityIdRef = useRef("1080p60"); + const picturePausedRef = useRef(false); const retiringStreamRef = useRef(null); const viewers = useMemo( @@ -143,6 +152,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { activeGenerationRef.current = null; generationRef.current += 1; sourceSwitchRef.current = null; + qualityChangeRef.current = null; signalRef.current?.stop(); peersRef.current.forEach((peer) => peer.dispose()); peersRef.current.clear(); @@ -163,6 +173,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { function disposeResources(notifyServer: boolean): void { sourceSwitchRef.current = null; + qualityChangeRef.current = null; const signal = signalRef.current; if (signal) { if (notifyServer) { @@ -187,6 +198,9 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { setPeerSnapshots(new Map()); setSignalStatus("offline"); setSwitchingSource(false); + setChangingQuality(false); + picturePausedRef.current = false; + setPicturePaused(false); } function forgetRoom(): void { @@ -226,7 +240,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { isCurrentGeneration(generation) && streamRef.current === captured ) { - endSharing("屏幕分享已结束"); + endSharing("已停止分享"); } }, { once: true }, @@ -241,6 +255,91 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { setSwitchingSource(false); } + function commitQuality(id: QualityProfileId): void { + qualityIdRef.current = id; + setQualityId(id); + } + + async function changeQuality(nextId: QualityProfileId): Promise { + if (phase !== "live") { + commitQuality(nextId); + return; + } + + const generation = activeGenerationRef.current; + const activeStream = streamRef.current; + if ( + generation === null || + !activeStream || + !isCurrentGeneration(generation) || + sourceSwitchRef.current || + qualityChangeRef.current + ) { + return; + } + + const token = {}; + qualityChangeRef.current = token; + setChangingQuality(true); + setNotice(null); + const profile = QUALITY_PROFILES[nextId]; + + try { + await applyCaptureProfile(activeStream, profile); + if ( + !isCurrentGeneration(generation) || + qualityChangeRef.current !== token || + streamRef.current !== activeStream + ) { + return; + } + + commitQuality(nextId); + setDetails(captureDetails(activeStream)); + const results = await Promise.all( + [...peersRef.current.values()].map((peer) => peer.updateProfile(profile)), + ); + if ( + isCurrentGeneration(generation) && + qualityChangeRef.current === token + ) { + const failed = results.filter((updated) => !updated).length; + setNotice( + failed > 0 + ? `画质已切换，${failed} 位观众未能应用新的发送参数` + : `画质已切换为 ${profile.label}`, + ); + } + } catch (error) { + if ( + isCurrentGeneration(generation) && + qualityChangeRef.current === token + ) { + setNotice(readableError(error)); + } + } finally { + if (qualityChangeRef.current === token) { + qualityChangeRef.current = null; + setChangingQuality(false); + } + } + } + + function togglePicturePause(): void { + const activeStream = streamRef.current; + if (phase !== "live" || !activeStream) { + return; + } + const nextPaused = !picturePausedRef.current; + if (!setVideoPaused(activeStream, nextPaused)) { + setNotice("当前分享没有可暂停的视频轨道"); + return; + } + picturePausedRef.current = nextPaused; + setPicturePaused(nextPaused); + setNotice(nextPaused ? "画面已暂停，音频不受影响" : "画面已恢复"); + } + function removePeer(peerId: string): void { peersRef.current.get(peerId)?.dispose(); peersRef.current.delete(peerId); @@ -280,7 +379,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { peerId, iceConfig, activeStream, - QUALITY_PROFILES[qualityId], + QUALITY_PROFILES[qualityIdRef.current], { sendSignal: (targetPeerId, payload) => isCurrentGeneration(generation) && signalRef.current === signal @@ -462,7 +561,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { let captured: MediaStream; try { // This must remain the first awaited operation in the button gesture. - captured = await captureDisplay(QUALITY_PROFILES[qualityId]); + captured = await captureDisplay(QUALITY_PROFILES[qualityIdRef.current]); } catch (error) { if (!isCurrentGeneration(generation)) { return; @@ -601,7 +700,8 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { phase !== "live" || generation === null || !isCurrentGeneration(generation) || - sourceSwitchRef.current + sourceSwitchRef.current || + qualityChangeRef.current ) { return; } @@ -614,7 +714,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { let captured: MediaStream; try { // Like initial capture, changing source must begin in this button gesture. - captured = await captureDisplay(QUALITY_PROFILES[qualityId]); + captured = await captureDisplay(QUALITY_PROFILES[qualityIdRef.current]); } catch (error) { if ( isCurrentGeneration(generation) && @@ -643,6 +743,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { } retiringStreamRef.current = previousStream; + setVideoPaused(captured, picturePausedRef.current); streamRef.current = captured; setStream(captured); setDetails(captureDetails(captured)); @@ -657,7 +758,7 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { peer, replaced: await peer.replaceStream( captured, - QUALITY_PROFILES[qualityId], + QUALITY_PROFILES[qualityIdRef.current], ), }; } catch { @@ -779,7 +880,22 @@ export function HostPage({ onAuthorizationRequired }: HostPageProps = {}) { + )} + {phase === "live" && ( + - ), - )} - - +
+
+ 画质 +
+ {(Object.keys(QUALITY_PROFILES) as QualityProfileId[]).map( + (id) => ( + + ), + )} +
+
+ {phase !== "live" && ( + )} + {phase !== "live" && ( -
- )} + )} + {room && (
diff --git a/src/client/pages/ViewerPage.tsx b/src/client/pages/ViewerPage.tsx index a7ee5cb..3403a40 100644 --- a/src/client/pages/ViewerPage.tsx +++ b/src/client/pages/ViewerPage.tsx @@ -387,7 +387,9 @@ export function ViewerPage({ roomId, onAuthorizationRequired }: ViewerPageProps)
- {!relayAvailable && signalStatus === "connected" && ( + {!relayAvailable && + signalStatus === "connected" && + (hostOnline || peerSnapshot) && ( TURN 未配置，严格网络可能无法连接 )} {peerSnapshot?.error && ( diff --git a/src/client/styles.css b/src/client/styles.css index 3375fb1..5cc02c6 100644 --- a/src/client/styles.css +++ b/src/client/styles.css @@ -724,6 +724,15 @@ h3 { grid-template-columns: minmax(0, 1fr) 330px; } + .section-heading { + flex-wrap: wrap; + } + + .broadcast-actions { + width: 100%; + flex-wrap: wrap; + } + .setup-controls { grid-template-columns: minmax(0, 1fr) auto; } @@ -777,18 +786,13 @@ h3 { } @media (max-width: 520px) { - .section-heading, .viewer-title-row { flex-wrap: wrap; } - .broadcast-actions { - width: 100%; - } - .broadcast-actions .button { min-width: 0; - flex: 1; + flex: 1 1 calc(50% - 4px); } .viewer-badges { diff --git a/src/client/webrtc/host-peer.ts b/src/client/webrtc/host-peer.ts index e8d7ece..aef4647 100644 --- a/src/client/webrtc/host-peer.ts +++ b/src/client/webrtc/host-peer.ts @@ -135,6 +135,28 @@ export class HostPeer { return true; } + async updateProfile(profile: QualityProfile): Promise { + if (this.disposed || !this.videoSender) { + return false; + } + try { + await configureVideoSender(this.videoSender, profile); + } catch (error) { + this.setError(error, "调整画质失败"); + return false; + } + if (this.disposed) { + return false; + } + this.profile = profile; + this.statsAccumulator.bytes = null; + this.statsAccumulator.frames = null; + this.statsAccumulator.timestamp = null; + this.snapshot = { ...this.snapshot, error: null }; + this.emit(); + return true; + } + async acceptSignal(payload: SignalPayload): Promise { if (this.disposed || payload.connectionId !== this.connectionId) { return; diff --git a/tests/host-peer-tracks.test.ts b/tests/host-peer-tracks.test.ts index b0432a0..0713085 100644 --- a/tests/host-peer-tracks.test.ts +++ b/tests/host-peer-tracks.test.ts @@ -5,7 +5,15 @@ import { HostPeer } from "../src/client/webrtc/host-peer.ts"; class FakeSender { failNextReplace = false; - readonly setParameters = vi.fn(async () => undefined); + failNextSetParameters = false; + readonly setParameters = vi.fn( + async (_parameters: RTCRtpSendParameters) => { + if (this.failNextSetParameters) { + this.failNextSetParameters = false; + throw new Error("setParameters failed"); + } + }, + ); readonly replaceTrack = vi.fn(async (track: MediaStreamTrack | null) => { if (this.failNextReplace) { this.failNextReplace = false; @@ -152,7 +160,7 @@ describe("HostPeer source replacement", () => { await expect( peer.replaceStream( createStream(createTrack("video", "next-video"), nextAudio), - QUALITY_PROFILES["720p60"], + QUALITY_PROFILES["1080p30"], ), ).resolves.toBe(true); expect(connection.senders[1]?.track).toBe(nextAudio); @@ -178,6 +186,54 @@ describe("HostPeer source replacement", () => { expect(connection.senders[1]?.track).toBeNull(); }); + it("updates quality parameters without replacing media tracks", async () => { + const video = createTrack("video", "video"); + const audio = createTrack("audio", "audio"); + const peer = createPeer(createStream(video, audio)); + + await expect(peer.start()).resolves.toBe(true); + const connection = FakePeerConnection.latest!; + await expect( + peer.updateProfile(QUALITY_PROFILES["1080p60"]), + ).resolves.toBe(true); + + expect(connection.senders[0]?.replaceTrack).not.toHaveBeenCalled(); + expect(connection.senders[1]?.replaceTrack).not.toHaveBeenCalled(); + expect(connection.senders[0]?.setParameters).toHaveBeenCalledTimes(2); + expect( + connection.senders[0]?.setParameters.mock.calls.at(-1)?.[0], + ).toMatchObject({ + degradationPreference: "balanced", + encodings: [{ maxBitrate: 8_000_000, maxFramerate: 60 }], + }); + }); + + it("can retry the selected quality after a sender update fails", async () => { + const peer = createPeer( + createStream( + createTrack("video", "video"), + createTrack("audio", "audio"), + ), + ); + + await expect(peer.start()).resolves.toBe(true); + const videoSender = FakePeerConnection.latest!.senders[0]!; + videoSender.failNextSetParameters = true; + + await expect( + peer.updateProfile(QUALITY_PROFILES["1080p60"]), + ).resolves.toBe(false); + await expect( + peer.updateProfile(QUALITY_PROFILES["1080p60"]), + ).resolves.toBe(true); + + expect(videoSender.setParameters).toHaveBeenCalledTimes(3); + expect(videoSender.setParameters.mock.calls.at(-1)?.[0]).toMatchObject({ + degradationPreference: "balanced", + encodings: [{ maxBitrate: 8_000_000, maxFramerate: 60 }], + }); + }); + it("rolls the first sender back when the second replacement fails", async () => { const oldVideo = createTrack("video", "old-video"); const oldAudio = createTrack("audio", "old-audio"); diff --git a/tests/quality.test.ts b/tests/quality.test.ts new file mode 100644 index 0000000..9e000e3 --- /dev/null +++ b/tests/quality.test.ts @@ -0,0 +1,90 @@ +import { afterEach, describe, expect, it, vi } from "vitest"; + +import { + applyCaptureProfile, + captureDisplay, + configureVideoSender, + QUALITY_PROFILES, + setVideoPaused, +} from "../src/client/media/quality.ts"; + +function createVideoStream() { + const videoTrack = { + contentHint: "", + enabled: true, + applyConstraints: vi.fn(async () => undefined), + } as unknown as MediaStreamTrack; + const stream = { + getTracks: () => [videoTrack], + getVideoTracks: () => [videoTrack], + getAudioTracks: () => [], + } as unknown as MediaStream; + return { stream, videoTrack }; +} + +afterEach(() => { + vi.unstubAllGlobals(); +}); + +describe("realtime quality controls", () => { + it("bounds initial capture to the selected profile", async () => { + const { stream, videoTrack } = createVideoStream(); + const getDisplayMedia = vi.fn(async () => stream); + vi.stubGlobal("navigator", { mediaDevices: { getDisplayMedia } }); + + await expect(captureDisplay(QUALITY_PROFILES["1080p30"])).resolves.toBe( + stream, + ); + + expect(getDisplayMedia).toHaveBeenCalledWith({ + video: { + width: { ideal: 1920, max: 1920 }, + height: { ideal: 1080, max: 1080 }, + frameRate: { ideal: 30, max: 30 }, + }, + audio: true, + }); + expect(videoTrack.contentHint).toBe("motion"); + }); + + it("changes capture constraints without selecting the source again", async () => { + const { stream, videoTrack } = createVideoStream(); + + await applyCaptureProfile(stream, QUALITY_PROFILES["720p30"]); + + expect(videoTrack.applyConstraints).toHaveBeenCalledWith({ + width: { ideal: 1280, max: 1280 }, + height: { ideal: 720, max: 720 }, + frameRate: { ideal: 30, max: 30 }, + }); + }); + + it("sets the screen-share sender degradation preference", async () => { + const parameters = { + encodings: [], + } as unknown as RTCRtpSendParameters; + const setParameters = vi.fn(async () => undefined); + const sender = { + getParameters: () => parameters, + setParameters, + } as unknown as RTCRtpSender; + + await configureVideoSender(sender, QUALITY_PROFILES["1080p60"]); + + expect(parameters.encodings[0]).toMatchObject({ + maxBitrate: 8_000_000, + maxFramerate: 60, + }); + expect(parameters.degradationPreference).toBe("balanced"); + expect(setParameters).toHaveBeenCalledWith(parameters); + }); + + it("pauses only the video track and can resume it", () => { + const { stream, videoTrack } = createVideoStream(); + + expect(setVideoPaused(stream, true)).toBe(true); + expect(videoTrack.enabled).toBe(false); + expect(setVideoPaused(stream, false)).toBe(true); + expect(videoTrack.enabled).toBe(true); + }); +}); From 74237639ccbb2fbef33e9e9826de4b40b44e3277 Mon Sep 17 00:00:00 2001 From: AAA <35992542+TNTcraftHIM@users.noreply.github.com> Date: Wed, 19 Aug 2026 02:23:15 +0800 Subject: [PATCH 2/2] docs: record live controls deployment (#15) --- docs/project-memory.md | 6 +++--- docs/status.md | 31 +++++++++++++++++++------------ 2 files changed, 22 insertions(+), 15 deletions(-) diff --git a/docs/project-memory.md b/docs/project-memory.md index 77fb9af..201862c 100644 --- a/docs/project-memory.md +++ b/docs/project-memory.md @@ -36,11 +36,11 @@ Last updated: 2026-08-19 - The repository contains a single npm package using Node.js 24, React, TypeScript, Vite, native WebRTC, `ws`, Zod, Vitest, and a separate coturn deployment. The server defaults to an all-interface listener for LAN development and containers; the bare-metal reverse-proxy baseline explicitly binds loopback. It provides a process-only `/healthz` endpoint and requires STUN plus explicit TURN/UDP and TURN/TCP URLs before production startup. TURN/TLS is accepted but optional; configuration checks do not establish public-network reachability. - The Web PoC implements capture-before-room creation, live source and profile changes without renegotiating healthy peers, three manual quality profiles with an explicit `balanced` sender preference, temporary picture pause with audio unaffected, one independent peer connection per viewer, stable signaling reconnect identities that also work for LAN viewers on HTTP, explicit ICE restart or peer rebuild, host session generation isolation, viewer connection-generation guards for asynchronous signaling and stats, short-lived TURN credentials, clearer waiting/TURN states, and local WebRTC statistics. - Access is deliberately small: `ACCESS_PASSWORD` is optional, site-wide, and accepts 1 through 128 visible ASCII characters when non-empty. When configured, host and viewer routes first establish a 12-hour stateless HMAC HttpOnly `SameSite=Strict` cookie; room creation and WebSocket upgrade accept that cookie and no direct room-creation Bearer bypass. There are no accounts, JWTs, server-side access-session maps, or logout flow. -- ADR-0002 accepts an optional `ROOM_DATABASE_PATH` only alongside `ACCESS_PASSWORD`. In that mode, built-in `node:sqlite` stores only an auto-incremented room ID and host-token digest; links do not expire and stopping sharing leaves the room waiting. Without the path, rooms remain random and temporary. Public mode can never use sequential persistent rooms. Commit `2f66770f8e90` is deployed with `/var/lib/screener/rooms.sqlite`; the state directory and database permissions are verified, and the empty database preserves ID `1` for the first real browser room. +- ADR-0002 accepts an optional `ROOM_DATABASE_PATH` only alongside `ACCESS_PASSWORD`. In that mode, built-in `node:sqlite` stores only an auto-incremented room ID and host-token digest; links do not expire and stopping sharing leaves the room waiting. Without the path, rooms remain random and temporary. Public mode can never use sequential persistent rooms. Commit `5b2fb005f6f7` is deployed with `/var/lib/screener/rooms.sqlite`; the state directory and database permissions are verified, and room ID `1` survived the deployment restart. - `/r/{code}` carries no viewer token or fragment, `/join` accepts only the numeric code, and the 256-bit host token stays internal to host authentication. In public mode the random code is the sole viewing capability and is not a strong privacy guarantee, so private Internet deployments should configure `ACCESS_PASSWORD`. - The Web control plane is deployed at `https://share.bonfire.icu` behind nginx with Node.js 24.19.0, and `turn.bonfire.icu` runs authenticated coturn 4.17.2 on standard UDP/TCP 3478. HTTPS, WSS, room authentication, certificate renewal, public STUN, authenticated TURN/UDP and TURN/TCP allocations, and relay-only bidirectional data paths are verified. TURN/TLS is intentionally not enabled. -- Automated checks pass 108 Vitest tests and both production builds; production remains at the earlier deployed revision. Same-machine synthetic-media Chromium recovery and public relay-only DataChannel evidence remains from that baseline. A real live quality/pause cycle, persistent-room stop/reuse/restart cycle, real screen/game audio, heterogeneous media sessions, mobile lifecycle handling, and latency or quality targets remain unverified. -- No infrastructure blocker remains for the current staging deployment. The immediate control-plane milestone and the separate real-device/media matrix are bounded in `docs/status.md`; neither justifies a native sender yet. +- Automated checks pass 108 Vitest tests and both production builds. Release `5b2fb005f6f7` passed loopback and public HTTPS/access-gate deployment checks with zero automatic service restarts; nginx, coturn, and the existing blog remained healthy. Same-machine synthetic-media Chromium recovery and public relay-only DataChannel evidence remain from the earlier baseline. A real live quality/pause cycle, persistent-room stop/reuse browser cycle, real screen/game audio, heterogeneous media sessions, mobile lifecycle handling, and latency or quality targets remain unverified. +- No infrastructure blocker remains for the current deployment. The live-control browser cycle and real-device media matrix are bounded in `docs/status.md`; neither justifies a native sender yet. ## Provisional Quality Targets diff --git a/docs/status.md b/docs/status.md index 2b825ac..6be422e 100644 --- a/docs/status.md +++ b/docs/status.md @@ -5,16 +5,15 @@ Last updated: 2026-08-19 ## Phase The WebRTC proof of concept is deployed at `https://share.bonfire.icu`. Commit -`2f66770f8e90` adds protected SQLite persistence, sequential room IDs, reusable -links, and a waiting state after sharing stops. The production database is -initialized and empty, so the first real persistent room can receive ID `1`. +`5b2fb005f6f7` adds live quality changes, picture pause, clearer connection +states, protected SQLite persistence, sequential room IDs, reusable links, and +a waiting state after sharing stops. The production database contains room ID +`1`; the deployment restart preserved that row. -The Web PoC now includes live quality changes, an explicit `balanced` sender -degradation preference, +The Web PoC includes an explicit `balanced` sender degradation preference, temporary picture pause with audio unaffected, and clearer stopped/waiting and -TURN-warning states. These controls are not deployed yet. Real capture, game -audio, mobile lifecycle, sustained multi-viewer behavior, and performance -targets remain unverified. +TURN-warning states. Real capture, game audio, mobile lifecycle, sustained +multi-viewer behavior, and performance targets remain unverified. ## Established Baseline @@ -30,6 +29,13 @@ targets remain unverified. ## Verification - `npm run check` passes type checking, 108 Vitest tests, and both production builds. +- Release `5b2fb005f6f7` passed a separate production-process smoke on loopback, + then was atomically activated under `/opt/screener/current`. External HTTPS + health, the protected access gate, authenticated Secure/HttpOnly cookie, and + the new 1080p30 client asset passed after restart. Screener reported zero + automatic restarts and no warning-level journal entries; nginx and coturn + remained active, while `bonfire.icu` and `www.bonfire.icu` continued returning + HTTP 200. - Headless Chromium 151 established real local peer connections from animated canvas streams, delivered a live video track over direct UDP, and recovered from dropped offer/answer and initial `createOffer` failures. - Host and viewer generation guards discard delayed signaling, candidate, and stats work after replacement. Synthetic source changes added and removed audio without another offer, preserved healthy peer objects, and stopped retired tracks. - Unit coverage verifies live capture constraints, balanced sender parameters, video-only pause, sender-update failure and retry, source replacement rollback, and audio `null -> track -> null` changes. @@ -39,9 +45,9 @@ targets remain unverified. ## Next Milestone -Merge and deploy the live controls, then verify a real quality/pause cycle, room -ID `1`, stop-and-republish behavior, reusable links, and recovery after a service -restart. +Verify a real quality/pause cycle in room ID `1`, stop-and-republish behavior, +link reuse in the browser, and host recovery after a service restart. The +database row itself has already survived the live-controls deployment restart. Execute and record the manual browser/network matrix: @@ -54,7 +60,8 @@ Execute and record the manual browser/network matrix: ## Current Blocker -- No infrastructure blocker remains. The live-control browser cycle, first real persistent-room lifecycle, and real-device media matrix remain unverified. +- No infrastructure blocker remains. The live-control browser cycle, room `1` + browser lifecycle, and real-device media matrix remain unverified. ## Blocking Decisions