From c8bc2fbee1dc33005fb38166dc0a6379f5acc98d Mon Sep 17 00:00:00 2001 From: AAA <35992542+TNTcraftHIM@users.noreply.github.com> Date: Wed, 19 Aug 2026 13:42:10 +0800 Subject: [PATCH] test(media): record clarity control evidence --- docs/project-memory.md | 12 ++-- docs/research/realtime-quality-adaptation.md | 61 +++++++++++--------- docs/status.md | 15 +++-- docs/方案设计.md | 2 + scripts/peer-assisted-benchmark.ts | 6 +- 5 files changed, 53 insertions(+), 43 deletions(-) diff --git a/docs/project-memory.md b/docs/project-memory.md index e72394d..59a9372 100644 --- a/docs/project-memory.md +++ b/docs/project-memory.md @@ -24,7 +24,7 @@ Last updated: 2026-08-19 - Keep user-operated mini-SFU deployment and central single-node SFU Draft PR #12 as optional infrastructure. When configured, the route controller may select them automatically as the last fallback; their fanout egress belongs to their operator and does not justify Redis or multi-node infrastructure. - Keep automatic routing behind ADR-0005: discrete failures drive peer recovery and optional SFU fallback, host media uses break-before-make and never exceeds two active edges, and a binary per-session relay capability keeps detected mobile/iPad clients as leaves. Corrected local recovery passes functionally but misses the sub-second gate. - Prefer direct UDP, then TURN/UDP, with TURN/TCP as the required non-UDP fallback. Optional TURN/TLS uses TCP 5349 by default; TCP 443 needs a dedicated address or validated L4/SNI routing. -- Treat 3/5/8 Mbps profiles as ceilings. Production users report severe degradation across all three; run one controlled 1/2/3-viewer capture before changing constants, then A/B clarity-first against `balanced`. Do not hide actual limits or add an automatic score/controller without evidence. +- Treat every quality setting as a ceiling. The current Draft defaults to clarity-first and adds bounded manual controls plus sender readback; production remains `balanced`, and its reported degradation is still unclassified. Run the 1/2/3-viewer plus TURN matrix before changing ceilings or adding any automatic controller. - Do not add custom scene detection or dynamic-FPS control until WebRTC statistics and host resource measurements prove a material gap. Keep browser codec order until target hardware proves a more efficient common codec. - Keep room policy deployment-driven. Public and password-only deployments use random temporary rooms. A site password plus SQLite path enables sequential persistent rooms; stopping sharing leaves the room and viewer link available. @@ -33,19 +33,19 @@ Last updated: 2026-08-19 - The repository is one npm package using Node.js 24, React, TypeScript, Vite, native WebRTC, `ws`, Zod, Vitest, and separate coturn. LAN/container listening defaults to all interfaces; bare-metal reverse-proxy deployment binds loopback and exposes `/healthz`. - Production commit `5b2fb005f6f7` is deployed at `https://share.bonfire.icu`. It provides live source and quality changes without rebuilding healthy peers, video-only pause, clearer connection states, and the standard one-host-`RTCPeerConnection`-per-viewer path. Production therefore still exceeds the two-edge host target above two viewers. - Production supports the three profiles with `contentHint = "motion"` and `degradationPreference = "balanced"`; its real live quality/pause cycle remains unverified. -- The stacked `feat/simple-entry-diagnostics` Draft puts equal share/join actions inside the idle 16:9 stage, defaults Host/Viewer technical details closed with one native checkbox, keeps actionable warnings outside it, and uses the bounded waiting/recovery/closed copy. Chrome 151 CDP checks pass at 320/375/390 CSS px without horizontal overflow; it is not deployed. +- The current quality stack includes Draft PR #19's equal idle-stage share/join actions and default-closed technical details; advanced quality stays in the common control area and actionable warnings stay visible. It is not deployed. - Access stays small: optional `ACCESS_PASSWORD`, a 12-hour stateless HMAC HttpOnly `SameSite=Strict` cookie, internal host token, role-bound signaling, Origin/payload checks, and no accounts, JWTs, or session map. - With `ROOM_DATABASE_PATH` and the site password, built-in SQLite stores only auto-incremented room ID and host-token digest. Links persist and stopping sharing leaves viewers waiting. Production room ID `1` survived the `5b2fb005f6f7` deployment restart. Without the path, rooms remain random and temporary; public mode cannot use sequential rooms. - Production runs behind nginx with Node.js 24.19.0. Authenticated coturn 4.17.2 at `turn.bonfire.icu:3478` has verified public STUN, TURN/UDP, TURN/TCP, and relay-only traffic. TURN/TLS is intentionally disabled. -- Draft PR #13 contains deterministic server/client peer-assisted assignment, standard remote-track relay, and synchronized 1080p60/1080p30/720p30 room profiles. It defaults off, is capped at eight viewers, re-encodes at every relay, and is neither merged nor deployed. Native shared encoding and custom encoded browser transport are not implemented. +- Draft PR #13 contains deterministic server/client peer-assisted assignment and standard remote-track relay. It defaults off, is capped at eight viewers, re-encodes at every relay, and is neither merged nor deployed. Native shared encoding and custom encoded browser transport are not implemented. - Draft PR #17 stacks on PR #13: peer recovery precedes an allowlisted SFU root; revisioned prepare/commit/abort is session-bound; active SFU failure gets one token refresh, then fails back and disables SFU for that share. LiveKit is dormant when unconfigured. - Corrected Chrome 151/LiveKit 1.13.5 smoke physically failed one leaf through peer recovery/reparent and two-root SFU. Frames resumed; 472 hook-assisted 25 ms samples saw host edge peak two. Failure-to-render took 2.257 seconds. It is localhost, headless, synthetic 720p30 video only. - Draft PR #16 (`5b09f0a`) proves one Pion RTP write fans equivalent data to two transports with independent SSRCs. It has no encoder and proves neither physical shared encode nor browser readiness. - Chrome 151 short synthetic `1/3/5/8` runs kept host edges at two, relay edges at one, and all viewers decoding; a three-viewer relay close recovered in 5.32 seconds. This proves topology/control only, not quality, load, endurance, or silent partitions. -- The peer-assisted authenticated snapshot and host-only update synchronize one bounded, non-persistent room profile. Online viewers receive changes; `ViewerRelay` uses the latest desired profile for current and future children, while `HostPeer` serializes sender setup, stream replacement, and last-wins profile updates. The ordinary P2P wire is unchanged. Real 1/3/5/8 quality, resource, and latency measurements remain pending. -- Chromium 151 kept relay/sender/signaling generations through 8 Mbps/60, 5 Mbps/30, and 3 Mbps/30 ceilings while the leaf decoded. Synthetic 640x360/30 proves control propagation, not full-resolution load. +- The current quality stack uses one strict, non-persistent `QualitySettings` value: 720p/1080p/1440p, integer 15-60 fps, 2-12 Mbps, and clarity/balanced/fluid preference. Host-only peer-assisted updates are last-wins for current/future relays and optional SFU; ordinary P2P wire stays unchanged. Sender updates are serialized and read back; rejection, rewriting, or three identical native limitation samples produce one visible warning without automatic action. +- Chrome 151 with one host, three viewers, and synthetic 720p30 passed balanced-to-clarity propagation, sender preference readback, stable peer-connection fingerprints, host fanout two, relay fanout one, and continued decoding. This is control evidence, not a quality, load, public-network, or endurance claim. - Per-frame encode/decode diagnostics use adjacent non-overlapping `getStats()` deltas rather than connection-lifetime averages; first, empty, changed-stream, and reset intervals remain unknown and rebase. -- Draft PR #17 passes CI, type checking, 19 Vitest files/253 tests, both builds, dependency audit, and repository hygiene. Production separately passed 108 tests and deployment checks. +- The current quality/UI stack passes repository hygiene, type checking, 20 Vitest files/260 tests, and both builds. Production separately passed 108 tests and deployment checks. ## Provisional Quality Targets diff --git a/docs/research/realtime-quality-adaptation.md b/docs/research/realtime-quality-adaptation.md index 8cd246d..76349cf 100644 --- a/docs/research/realtime-quality-adaptation.md +++ b/docs/research/realtime-quality-adaptation.md @@ -6,17 +6,16 @@ ## Finding -Screener should not optimize a single quality dimension at the expense of the -others. The previous combination of `contentHint = "motion"` and an explicit -`degradationPreference = "maintain-framerate"` told WebRTC to preserve frame -rate primarily by reducing resolution. That is appropriate for some fast -motion, but it can make game UI, maps, subtitles, and text unreadable. +Screener should not optimize one quality dimension at the expense of the +others. `maintain-framerate` may preserve motion by reducing resolution until +game UI, maps, subtitles, and text become unreadable; `maintain-resolution` +may instead lower frame rate. Neither preference overrides congestion control. -The smallest correction is to retain the game-oriented `motion` hint while -explicitly selecting `balanced` degradation. The WebRTC API allows the user -agent to balance frame-rate and resolution degradation; it does not prescribe -the algorithm or guarantee a clarity-first result. Congestion control remains -authoritative, and Chromium, Firefox, and Safari behavior must be measured. +The current bounded implementation retains the game-oriented `motion` hint, +defaults to `maintain-resolution`, and offers explicit balanced and fluid +choices. This is a readability-first default, not a quality guarantee: the +WebRTC API describes a user-agent preference rather than its algorithm, so +Chromium, Firefox, and Safari behavior still needs measured comparison. The inspected Chromium/libwebrtc source chain makes a screen-only shortcut especially unsafe to assume for Screener. The JavaScript `motion` hint reaches @@ -102,28 +101,37 @@ resolution, frame rate, or bitrate. - A live profile change uses `track.applyConstraints()` and updates every current sender with `RTCRtpSender.setParameters()`. It does not reopen the source picker or renegotiate healthy peer connections. -- The video track keeps `contentHint = "motion"`, while each sender explicitly - uses `degradationPreference = "balanced"`. This is a user-agent preference, - not a promise to preserve either resolution or frame rate. +- The video track keeps `contentHint = "motion"`; recommended profiles default + to `maintain-resolution`, with explicit `balanced` and + `maintain-framerate` choices. None promises an emitted resolution or rate. - `maxBitrate` and `maxFramerate` are ceilings. They are neither minimums nor target guarantees, and the project does not use SDP bitrate hacks. +- The folded advanced panel accepts only 720p/1080p/1440p, integer 15-60 fps, + 2-12 Mbps, and the three preferences. Audio stereo/bitrate is not exposed + because standard sender controls cannot reliably guarantee the negotiated + browser audio mode. +- Every sender update derives from `getParameters()`, calls `setParameters()`, + then reads requested/applied bitrate, frame rate, scale, and preference. + Rejection or browser rewriting is visible rather than console-only. +- One strict room setting is last-wins for current/future peer relays and the + optional SFU publisher. Ordinary P2P keeps that state local and does not add + it to the authenticated wire. - Pausing the picture disables the existing video track, producing black video without closing the room or media connection. Audio remains enabled. -The next bounded A/B changes only degradation preference: keep `motion`, compare -the current `balanced` behavior with `maintain-resolution`, and expose the -result as a default clarity-first choice plus optional balanced/fluid choices. -It may trade frame rate for readability and is not a universal fix. The -advanced panel remains bounded to resolution ceiling, frame-rate ceiling, -bitrate ceiling, quality priority, and audio mode; it does not expose raw SDP or -codec internals. +The implementation deliberately stops at manual bounded controls. It adds no +composite score, periodic adjustment, codec forcing, SDP bitrate manipulation, +or scene detector. Three consecutive samples of one non-`none` native +`qualityLimitationReason` produce one explanatory warning; a reason change or +recovery resets it and never triggers a media action. -A Chromium 151 loopback smoke with one host, three viewers, and a synthetic -640x360/30 source kept the same relay peer, sender, and signaling generations -while applying 8 Mbps/60 fps, 5 Mbps/30 fps, and 3 Mbps/30 fps sender ceilings. -The leaf continued decoding after both live switches. This verifies control -propagation and connection preservation, not actual 1080p output, visual -quality, CPU/GPU cost, public-network behavior, or sustained performance. +A Chromium 151 loopback smoke with one host, three viewers, and synthetic +720p30 video propagated balanced then clarity settings to every participant, +displayed both sender preference readbacks, and preserved all peer-connection +fingerprints. Host media edges peaked at two, relay edges at one, and all three +viewers kept decoding. This verifies controls and connection preservation, not +visual quality, full-resolution performance, CPU/GPU cost, public networks, or +sustained behavior. ## Deliberate Non-Goals @@ -166,6 +174,7 @@ profiles before adding an application-level adaptation controller. - [W3C Screen Capture](https://www.w3.org/TR/screen-capture/) - [W3C WebRTC](https://www.w3.org/TR/webrtc/) - [W3C WebRTC Statistics](https://www.w3.org/TR/webrtc-stats/) +- [MDN `RTCRtpSender.setParameters()`](https://developer.mozilla.org/en-US/docs/Web/API/RTCRtpSender/setParameters) - [Chromium `motion` to libwebrtc `kFluid` bridge](https://chromium.googlesource.com/chromium/src/third_party/+/refs/heads/main/blink/renderer/modules/peerconnection/media_stream_video_webrtc_sink.cc) - [libwebrtc `motion`/`kFluid` sender classification](https://webrtc.googlesource.com/src/+/3b1eab8a69cb5078befb021c5492d3f204a7d6a2/pc/rtp_sender.cc) - [libwebrtc encoder content-type selection](https://webrtc.googlesource.com/src/+/9caef2a8b88f389af10cee841732c42a98d3d45d/media/engine/webrtc_video_engine.cc) diff --git a/docs/status.md b/docs/status.md index 4be93d6..4e040b3 100644 --- a/docs/status.md +++ b/docs/status.md @@ -21,19 +21,18 @@ functionally but misses its sub-second gate; neither Draft is merged or deployed - Direct ICE is preferred independently per media edge. Authenticated TURN/UDP and TURN/TCP are required production fallbacks; TURN/TLS is optional. - Hidden routing is `direct P2P -> peer-assisted -> optional SFU`. After bounded ICE recovery it tries peer reparenting before an allowlisted SFU root. Session-bound revisions prepare, commit break-before-make under two host edges, or abort. Active SFU gets one token refresh, then fails back for that share. - A viewer starts as a leaf each session and explicitly advertises relay capacity zero or one; the Web client reports detected mobile/iPad clients as zero and desktop-class browsers as one. Withdrawal stops future assignment without moving a healthy edge. Browser relays remain one-child; the host remains two-child. -- Peer-assisted profile state is bounded, memory-only, absent from ordinary P2P wire, and applied to current and future relay children. -- The stacked `feat/simple-entry-diagnostics` Draft places equal share/join actions in the idle 16:9 stage, removes the duplicate weak entry, and defaults Host/Viewer technical details closed while keeping actionable warnings visible. It is not deployed. +- The quality stack replaces the three-ID relay state with one strict, memory-only setting shared by current/future relays and optional SFU; ordinary P2P wire stays unchanged. It defaults clarity-first and exposes bounded manual ceilings. +- The same stack includes Draft PR #19's equal idle-stage share/join actions and default-closed technical details; actionable warnings remain visible. It is not deployed. ## Verified Evidence - Production passed 108 tests, both builds, public HTTPS/access/TURN checks, clean activation, and a database restart retaining room `1`. - Draft PR #13 now provides `npm run benchmark:peer-assisted`; the full check passes type checking, 13 Vitest files with 152 tests, and both production builds. - A short Chrome 151 synthetic `1/3/5/8` benchmark passed every topology check: host active edges peaked at two, relay edges at one, every viewer kept increasing decoded frames through the measurement window, and the slowest first decoded frame was about 1.05 seconds. Closing a first-level relay in the three-viewer run recovered in about 5.32 seconds without exceeding host fanout two. -- A separate live-profile smoke kept the same relay peer, sender, and signaling generations through 8 Mbps/60, 5 Mbps/30, and 3 Mbps/30 ceilings while its leaf kept decoding. +- Chrome 151 with one host, three viewers, and synthetic 720p30 passed balanced-to-clarity propagation, visible sender preference readback, unchanged peer-connection fingerprints, host fanout two, relay fanout one, and continued decoding. - Corrected Chrome 151/LiveKit 1.13.5 synthetic 720p30 smoke physically failed the same leaf through peer recovery/reparent and a two-root SFU route. Its frames resumed; 472 hook-assisted 25 ms samples saw host edge peak two. Failure report to active took 1.481 seconds and to new render 2.257 seconds. -- Draft PR #17 passes CI, type checking, 19 Vitest files/253 tests, both production builds, dependency audit, and repository hygiene. +- Draft PR #17 passed its 253-test baseline; the current quality/UI stack passes repository hygiene, type checking, 20 Vitest files/260 tests, and both production builds. - Chrome 151 CDP checks at 320/375/390 CSS px keep the two idle-stage actions equal, on one row, 44 px high, and free of horizontal overflow. The Viewer waiting page also has no overflow; its details checkbox starts false, changes locally, and resets after navigation. These checks cover idle/waiting states, not live media. -- The stacked UI branch passes type checking, 20 Vitest files/254 tests, both production builds, and the production dependency audit. - Draft PR [#16](https://github.com/TNTcraftHIM/Screener/pull/16) passes CI for one Pion RTP write fanned to two transports. It has no encoder and proves neither physical encode nor browser E2E. - Local diagnostics use adjacent non-overlapping `getStats()` deltas; empty, changed-stream, and reset intervals rebase instead of publishing lifetime averages. - Production HTTPS/WSS, access cookie, room/WebSocket authorization, certificate renewal, public STUN, and authenticated TURN/UDP and TURN/TCP relay-only bidirectional paths are verified. TURN/TLS is intentionally disabled. @@ -44,7 +43,7 @@ functionally but misses its sub-second gate; neither Draft is merged or deployed - Android Chrome and iOS Safari remain unverified leaves. Runtime capability conservatively marks detected mobile/iPad clients as leaves; real UA/lifecycle behavior remains open. - The observed recovery starts from a page close immediately seen by the server. A silent partition can wait 30 to 60 seconds for heartbeat detection before the default 5-second grace; it remains unverified. - Real screen/game audio, heterogeneous machines and networks, mobile lifecycle behavior, the production live quality/pause cycle, room `1` stop-and-republish/link reuse, and sustained profile performance remain unverified. -- Production users report severe resolution/bitrate/FPS degradation across all profiles. A controlled 1/2/3-viewer and TURN sample must distinguish capture, per-edge CPU, bandwidth/path, and receiver limits before changing ceilings or codecs; clarity-first versus `balanced` is the first bounded A/B. +- Production users report severe resolution/bitrate/FPS degradation across all profiles. The Draft now permits bounded clarity/balanced comparison, but a controlled 1/2/3-viewer and TURN sample must still distinguish capture, per-edge CPU, path, and receiver limits before any quality claim or automatic controller. - Browser relays do not share encoding. Encoded objects, custom congestion control, multiple trees, and network coding remain separate measured candidates. - The long-term endpoint budget is at most two downstream edges for both the host and relay-capable viewers, with one compatible encoded output reused across both edges where a native media engine can prove it. The current browser spike remains host capacity two/viewer capacity one and performs a new encode at each relay. Packet/layer striping and multi-parent assembly are recorded, not implemented. - The corrected smoke is localhost/headless/video-only; public transport, audio, shaping, load, mobile, endurance, and sub-25 ms overlap remain open. @@ -52,8 +51,8 @@ functionally but misses its sub-second gate; neither Draft is merged or deployed ## Next Milestone Finish ADR-0005: address the 2.257-second fallback, then validate public -transports, rollback, reconnect, controls, edge counts, egress, and load. Keep -the stacked entry/diagnostics Draft separate and unmerged until its parent lands. +transports, rollback, reconnect, controls, edge counts, egress, and load. The +quality/UI stack remains undeployed until its parent gates pass. ADR-0004 still requires a full-resolution 30-minute `1/3/5/8` network, resource, quality, latency, recovery, and browser/mobile-leaf matrix. diff --git a/docs/方案设计.md b/docs/方案设计.md index b80d025..14d548d 100644 --- a/docs/方案设计.md +++ b/docs/方案设计.md @@ -218,6 +218,7 @@ TURN/TLS 是可选能力，不是生产启动门槛。启用时默认使用 RFC - SQLite 房间库从 `1` 自增、重启恢复、持久房间无过期时间、数据最小化、数据库路径无全站密码时启动失败； - TURN HMAC 固定向量、TTL、生产环境缺少 STUN 或 TURN/UDP+TCP 覆盖时启动失败，并接受可选 TURN/TLS； - 协议非法 JSON、未知消息、错误方向、跨房间目标和 payload 限制； +- 质量协议的 host 授权、严格字段与数值范围、last-wins、当前/未来 relay、可选 SFU 和普通 P2P wire 兼容；sender 的四字段 requested/applied 读回、拒绝与持续受限提示； - HTTP/WS 集成：密码模式下 host/viewer gate、建房和 upgrade 只认 cookie 且无建房 Bearer 旁路；一名 host、多名 viewer、定向 offer/answer/candidate、连接上限、grace 内恢复、显式 restart/rebuild、离开和停止分享后的等待状态； - 客户端策略与 stats 解析：会话替换不重连，本地 TURN protocol 与 ICE protocol 分离，远端 TURN transport 保持未知； - TypeScript 类型检查、Vite production build 和仓库 hygiene。 @@ -230,6 +231,7 @@ TURN/TLS 是可选能力，不是生产启动门槛。启用时默认使用 RFC - Windows Chrome/Edge 的音轨有/无； - Android Chrome 与 iOS Safari 的播放手势、旋转、前后台和网络切换； - 一名分享者、八名异构观看者连续 30 分钟，同时记录较低人数基线、实际上行、编码负载和首帧时间。真实 1:8 未完成前不得把默认上限写成性能承诺。 +- 真实桌面 Chrome 对 balanced 与 clarity 做控制传播和连接身份 smoke；该项只证明直播中应用与不重建，不代替清晰度、负载或公网矩阵。 真实矩阵未完成前，状态文档必须把这些列为待验证，而不是把设计目标写成产品能力。 diff --git a/scripts/peer-assisted-benchmark.ts b/scripts/peer-assisted-benchmark.ts index c220812..525143f 100644 --- a/scripts/peer-assisted-benchmark.ts +++ b/scripts/peer-assisted-benchmark.ts @@ -1245,7 +1245,7 @@ async function startHost( await waitForPage( cdp, page, - "document.querySelector('.start-button')", + "document.querySelector('.entry-actions button.entry-action')", 15_000, "host controls", signal, @@ -1254,11 +1254,11 @@ async function startHost( cdp, page, `(() => { - const profiles = document.querySelectorAll('.segmented-control button'); + const profiles = document.querySelectorAll('.quality-controls .segmented-control button'); const profile = profiles[${profileIndex}]; if (!(profile instanceof HTMLButtonElement)) throw new Error('Quality profile button missing'); profile.click(); - const start = document.querySelector('.start-button'); + const start = document.querySelector('.entry-actions button.entry-action'); if (!(start instanceof HTMLButtonElement)) throw new Error('Start button missing'); start.click(); return true;