docs(media): fence codec proof attempts (#219)
docs(media): fence codec proof attempts
This commit is contained in:
AAA authored and GitHub committed
commit
dfa2b071e2
6 files changed
+287
-4
No files matched your search
@@ -18,6 +18,7 @@
|
||||
- [Native H.264 hardware decision](./research/native-h264-hardware-decision.md): bounded WebCodecs no-go, proved Media Foundation/NVIDIA hardware encode, the default-fmtp `42c01f` stop, and the retained interop boundary.
|
||||
- [Native H.264 opt-in path](./research/native-h264-opt-in-path.md): explicit H.264 sender mode, exact Pion profile registration, and one bounded Host/Viewer loopback with hardware caveats.
|
||||
- [Realtime quality adaptation](./research/realtime-quality-adaptation.md): diagnostic-only capture/send/receive correlation, production generation-rebuild evidence, path-isolated dual representations, and bounded simulcast/Dynacast/SVC checks.
|
||||
- [Browser background capture diagnostics](./research/browser-background-capture.md): browser lifecycle evidence, a privacy-bounded capture/send/receive experiment, interpretation gates, and non-goal keepalive mechanisms.
|
||||
- [Browser screen-audio quality](./research/browser-screen-audio-quality.md): capture/source compatibility, standard readback limits, Opus negotiation boundaries, and the retained no-control decision.
|
||||
- [Display name and presence boundaries](./research/display-name-presence.md): browser-local storage failure behavior, Unicode normalization, control-character policy, and the Web-only presence capability boundary.
|
||||
- [Agent context governance research](./research/agent-context-governance.md): official Codex, Claude Code, Hermes Agent, and GitHub practices adopted by this repository.
|
||||
|
||||
@@ -18,7 +18,7 @@ Last updated: 2026-08-24
|
||||
- Ordinary peer ICE is STUN-only, and current LiveKit media is UDP-only. Credentials, room secrets, candidate details, and diagnostic data remain private and narrowly scoped. A future strict-firewall transport requires separate evidence and must remain inside LiveKit rather than becoming another Screener route candidate.
|
||||
- Browser peers use existing periodic WebRTC/LiveKit stats to maintain one exact-route decoded-progress timestamp. New frames or route/connection changes reset it; a non-paused deadline reports the edge once. Track availability, bitrate, FPS, resolution, freeze, RTT, jitter, loss, sender stats, and SFU layer describe only the current path: they remain framework or diagnostic inputs and cannot prove another parent would be better.
|
||||
- Browser capture already requests available share audio by default on initial capture and source switch, with an explicit video-only warning when no audio track is returned. The accepted v9 screen-audio control offers live-switchable 64/128/256 kbps sender ceilings and defaults to 128. The existing last-wins quality wire owns desired state; each endpoint serially applies and reads back current Host/relay/SFU audio senders, exposes local failures, and gives future senders the latest desired value. Opus remains fixed, and without an applied-ack wire the Host does not claim room-wide atomic convergence. These are configuration ceilings, not fidelity claims.
|
||||
- Video codec selection remains `automatic | H.264 | VP8`. The accepted first switching boundary is a generation-fenced renegotiation/republish transaction while sharing is explicitly paused. Preparation never resumes a user-paused share: it stops its deadline but retains the single room-serial codec owner as one bounded prepared record with no timer. The user's normal Resume atomically revalidates those exact bindings inside that owner, starts the proof deadline, and only then enables the source. Proof failure first restores authoritative pause and immediately runs bounded rollback preparation; the resulting prepared rollback has no timer and waits for the user's next normal Resume before revalidation, proof, and source enable. Unpaused hot switching is later reuse of that transaction, not an independent mechanism.
|
||||
- Video codec selection remains `automatic | H.264 | VP8`. The accepted first switching boundary is a generation-fenced renegotiation/republish transaction while sharing is explicitly paused. Preparation never resumes a user-paused share: it stops its deadline but retains the single room-serial codec owner as one bounded prepared record with no timer. The user's normal Resume atomically revalidates those exact bindings inside that owner, allocates the next server-monotonic proof attempt, starts the proof deadline, and only then enables the source. Proof request and response echo that exact attempt, and Pause invalidates it before another Resume can allocate the next attempt. Proof failure first restores authoritative pause and immediately runs bounded rollback preparation; the resulting prepared rollback has no timer and waits for the user's next normal Resume before revalidation, proof, and source enable. Unpaused hot switching is later reuse of that transaction, not an independent mechanism.
|
||||
- Recommended quality remains exactly `1080p60`, `1080p30`, and `720p30`, with the middle `1080p30` as the accepted default. `480p` means only an advanced `854x480` resolution choice whose FPS and bitrate remain independently selected; it is not a fourth recommended profile.
|
||||
- Two distribution artifacts are accepted only after the current feature, NAT, and real-network ledger is complete. A public-server one-click deployment package installs the current application plus STUN/SFU and proxy components on a user-owned server. A Windows/macOS/Linux local package runs the Host, application server, and local state on the broadcaster's machine without source, Node, or any external Screener/network service; it keeps workable direct P2P but reports the reachability loss imposed by public ingress, TLS, gateway, NAT, or firewall limits rather than promising universal connectivity.
|
||||
- Current UI, presence, route labels, and diagnostics describe observed state only; they do not create route authority.
|
||||
|
||||
@@ -0,0 +1,282 @@
|
||||
# Browser Background Capture Diagnostics
|
||||
|
||||
- Research date: 2026-08-24
|
||||
- Scope: desktop Chrome/Edge Web Host capture while the Host page is
|
||||
unfocused, occluded, backgrounded, or minimized
|
||||
- Status: diagnostic plan; the reported degradation has not been reproduced or
|
||||
attributed under controlled measurement, and no fix is accepted, implemented,
|
||||
or deployed
|
||||
|
||||
## Current Conclusion
|
||||
|
||||
There is no evidence that Screener is missing a page-liveness signal. A live
|
||||
display-capture track and a live WebRTC connection already put the page in the
|
||||
media cases that current Chrome excludes from intensive timer throttling and
|
||||
Energy Saver freezing. Those exclusions do not guarantee a requested capture
|
||||
rate, encoder throughput, or remote rendering rate, and no Web API lets an
|
||||
application assert foreground scheduling priority.
|
||||
|
||||
The current report therefore remains diagnostic-only. It must distinguish five
|
||||
different effects before any product change is considered:
|
||||
|
||||
1. Host preview or stats presentation is throttled while transmitted media is
|
||||
unchanged.
|
||||
2. The selected display surface becomes inaccessible, muted, unchanged, or
|
||||
otherwise constrained by the browser, operating system, or capture backend.
|
||||
3. Capture work or video encoding is CPU/GPU constrained.
|
||||
4. WebRTC congestion control reduces outbound resolution, frame rate, or
|
||||
bitrate.
|
||||
5. The Viewer receives the normal stream but decodes, renders, or displays it
|
||||
differently.
|
||||
|
||||
The Host page lifecycle and the captured source lifecycle are independent
|
||||
variables. The Screen Capture specification permits a user agent to treat a
|
||||
minimized captured surface as temporarily inaccessible and mute its track; it
|
||||
does not say that minimizing the separate capturing page must reduce capture.
|
||||
Capture constraints are post-selection preferences and allow frame decimation,
|
||||
so requested `maxFramerate` is not proof of actual source or encoded frame rate.
|
||||
|
||||
The current default video preference is `automatic`, not a contractual VP8
|
||||
selection. Every run must record the actual negotiated codec and encoder.
|
||||
H.264, VP8, and encoder implementation are diagnostic variables only after a
|
||||
stable baseline exists; none is a page-keepalive mechanism.
|
||||
|
||||
## Current Source Boundary
|
||||
|
||||
The source review used `main` commit
|
||||
`64755924c893b4f09b24f81c9a81891cc050ed44`. The relevant capture, preview,
|
||||
sender, and stats source is unchanged from production release
|
||||
`8f5b3f192ddd010ca01c969008e512191312736a`.
|
||||
|
||||
- `src/client/media/quality.ts` obtains one `getDisplayMedia()` stream, applies
|
||||
ideal/max capture constraints, marks video as `motion`, and applies sender
|
||||
bitrate, frame-rate, and degradation ceilings. Only an explicit sharing
|
||||
pause changes capture tracks' `enabled` state.
|
||||
- `src/client/pages/HostPage.tsx` pauses the existing local preview video when
|
||||
the Host document is hidden or unfocused and resumes that preview when it is
|
||||
visible and focused. This does not stop, mute, disable, or replace the capture
|
||||
track or any sender.
|
||||
- No `requestAnimationFrame()` loop drives production capture or sending.
|
||||
WebRTC capture, encoding, congestion control, and transport remain
|
||||
browser-owned.
|
||||
- `src/client/webrtc/stats.ts` already distinguishes capture settings,
|
||||
`media-source` encoder-input rate, outbound RTP rate and counters, encode
|
||||
work, limitation reason, codec, encoder, and receiver metrics. Its interval
|
||||
calculations use RTCStats timestamps, so delayed UI timer delivery must not
|
||||
be inferred as an equal media-rate drop.
|
||||
- `src/client/lib/diagnostic-export.ts` exports only the latest bounded sample.
|
||||
It is a useful cross-check, not a historical trace.
|
||||
- `scripts/peer-assisted-benchmark.ts` is not evidence for this issue: its
|
||||
synthetic canvas source is timer-driven and its browser launch explicitly
|
||||
disables background timer, occluded-window, and renderer backgrounding.
|
||||
|
||||
The first investigation uses these existing fields plus local browser
|
||||
diagnostics. It does not authorize a server telemetry schema, persistent
|
||||
history, background push, or another media lifecycle.
|
||||
|
||||
## Primary Evidence
|
||||
|
||||
All sources were checked on 2026-08-24.
|
||||
|
||||
- [W3C Screen Capture](https://www.w3.org/TR/screen-capture/) defines the
|
||||
captured-surface mute/end lifecycle and permits post-selection downscaling
|
||||
and frame decimation. It leaves the definition of temporary surface
|
||||
inaccessibility to the user agent and operating system. This is a Working
|
||||
Draft, so it is a diagnostic boundary rather than a cross-browser performance
|
||||
guarantee.
|
||||
- [W3C Media Capture and Streams](https://www.w3.org/TR/mediacapture-streams/)
|
||||
keeps a `MediaStreamTrack` and its consumers distinct. A local media element
|
||||
and an `RTCPeerConnection` consume the same track independently.
|
||||
- [W3C WebRTC Statistics](https://w3c.github.io/webrtc-stats/) defines
|
||||
`media-source` as media after track constraints and before encoding, while
|
||||
outbound RTP describes encoded transmission. Cumulative counter deltas and
|
||||
stats timestamps are the comparison basis. Optional absent fields remain
|
||||
unknown. This is an Editor's Draft and does not require every browser to emit
|
||||
every optional member.
|
||||
- [W3C Page Visibility](https://www.w3.org/TR/page-visibility-2/) defines
|
||||
observable document visibility; it does not define capture or encoder
|
||||
priority.
|
||||
- [Chrome chained-timer throttling](https://developer.chrome.com/blog/timer-throttling-in-chrome-88)
|
||||
says hidden-page timers may be batched and `requestAnimationFrame()` is not a
|
||||
background clock. A live WebRTC media track avoids the intensive once-per-
|
||||
minute timer tier, but ordinary background timer batching can still make a
|
||||
UI sampler appear stale.
|
||||
- [Chrome Energy Saver freezing](https://developer.chrome.com/blog/freezing-on-energy-saver)
|
||||
excludes active screen/window/tab capture and a peer connection with a live
|
||||
media track from the documented Chrome 133 freezing policy. That exclusion
|
||||
is not an encoder or frame-rate guarantee.
|
||||
- [Chromium `DesktopCaptureDevice`](https://chromium.googlesource.com/chromium/src/+/master/content/browser/media/capture/desktop_capture_device.cc)
|
||||
schedules capture in the browser capture subsystem, requests an internal
|
||||
display-sleep wake lock, and by default limits desktop capture to 50% of one
|
||||
CPU core by lengthening the next capture period when capture work is slow.
|
||||
This is direct evidence for measuring capture cost rather than manufacturing
|
||||
renderer activity.
|
||||
- [Chromium WebRTC internals owner](https://chromium.googlesource.com/chromium/src/+/master/content/browser/webrtc/webrtc_internals.cc)
|
||||
requests an internal application-suspension wake lock for active peer
|
||||
connections. This internal browser behavior is not equivalent to the Web
|
||||
Screen Wake Lock API and is not an application control surface.
|
||||
- [Chromium WebRTC internals dump UI](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/content/browser/webrtc/resources/webrtc_internals.html)
|
||||
provides local API-event and getStats evidence. Its full dump is private lab
|
||||
input, not a project artifact.
|
||||
- [W3C Screen Wake Lock](https://www.w3.org/TR/screen-wake-lock/) only attempts
|
||||
to prevent a visible document's screen from turning off; hidden documents
|
||||
lose the lock. It does not promise capture, encoder, timer, or renderer
|
||||
priority.
|
||||
|
||||
## Minimal Reproduction
|
||||
|
||||
### Baseline
|
||||
|
||||
Use one desktop Host and one separate physical Viewer on a direct, wired path.
|
||||
Do not include SFU, peer relay, multiple senders, Wi-Fi, or cellular behavior in
|
||||
the first baseline. Use a supported stock Chrome/Edge build with a clean
|
||||
profile and no background-throttling flags. Record browser and OS versions,
|
||||
power source, Energy Saver state, display topology, and coarse CPU/GPU load.
|
||||
Exact device, driver, and hardware identifiers remain local.
|
||||
|
||||
Use a deterministic, continuously changing game scene or built-in replay on a
|
||||
real display surface. A JavaScript canvas animation is not suitable because
|
||||
its own timer or animation-frame lifecycle would confound the capture result.
|
||||
Keep the selected source foreground, visible, and changing throughout the
|
||||
first matrix.
|
||||
|
||||
Open `chrome://webrtc-internals` on Host and Viewer before sharing and keep its
|
||||
overhead constant across every run. Leave audio-debug recording, event-log
|
||||
recording, packet capture, and media recording disabled. The current Screener
|
||||
diagnostic download may be taken at state boundaries as a latest-sample
|
||||
cross-check; it is not used as a time series.
|
||||
|
||||
### Host-State Matrix
|
||||
|
||||
Run these Host-page states while the captured source, Viewer, route, scene, and
|
||||
quality setting remain unchanged:
|
||||
|
||||
| State | Host document | Browser window | Captured source |
|
||||
| --- | --- | --- | --- |
|
||||
| Focused | visible and focused | visible | visible and changing |
|
||||
| Unfocused | visible, not focused | visible or occluded | visible and changing |
|
||||
| Background tab | hidden | visible | visible and changing |
|
||||
| Minimized | hidden | minimized | visible and changing on another display |
|
||||
|
||||
Use equal 60-second measurement windows, restore the focused baseline between
|
||||
states, randomize state order, and repeat three fresh controlled runs. These
|
||||
window and repeat counts define a small comparable sample; they are not product
|
||||
timeouts, quality thresholds, or proof of statistical significance. Record the
|
||||
actual initial negotiated codec and encoder with `automatic`; do not switch
|
||||
codec inside a run.
|
||||
|
||||
For each exact overlapping window, correlate:
|
||||
|
||||
| Stage | Required evidence |
|
||||
| --- | --- |
|
||||
| A. Capture | track `enabled`/`muted`/`readyState`; `displaySurface`; `getSettings()` width, height, and requested-result frame rate; `RTCVideoSourceStats` frames and frame rate; Host visibility/focus and local preview `paused` state |
|
||||
| B. Sender | outbound `framesEncoded`, frame rate, bytes, frame size, `totalEncodeTime` delta, `qualityLimitationReason`, target/available bitrate when present, actual codec/profile, encoder implementation, retransmission, loss, and RTT |
|
||||
| C. Viewer | inbound bytes and frame rate, frames received/decoded/dropped, freeze count/duration deltas, loss, jitter, actual codec, and foreground render behavior |
|
||||
|
||||
Use RTCStats object identity, media generation, and timestamps to form adjacent,
|
||||
non-overlapping deltas. A missing optional member, changed stats object, counter
|
||||
reset, or non-overlapping interval remains unknown rather than zero. Record OS,
|
||||
browser, game, and GPU/CPU observations over the same timestamps; do not infer a
|
||||
media result from the Host UI refresh cadence.
|
||||
|
||||
### Follow-Up Only After Stable Reproduction
|
||||
|
||||
Change one variable per fresh run:
|
||||
|
||||
1. Compare entire-monitor and application-window capture, then vary the
|
||||
captured source's own visible, occluded, and minimized state while the Host
|
||||
page state stays fixed.
|
||||
2. Compare local preview playing and paused while capture and sender identities
|
||||
stay fixed.
|
||||
3. Run separate new Host/share/peer-connection sessions for `automatic`, H.264,
|
||||
and VP8, and verify the actual codec and encoder in each session.
|
||||
4. When investigating the 2026-08-19 production report, compare exact release
|
||||
`769de201f7cc` with the then-current exact `main` commit on the same machine,
|
||||
browser, driver, game scene, and direct wired Viewer.
|
||||
|
||||
Do not run a full factorial matrix before the baseline reproduces. A follow-up
|
||||
must have one stated hypothesis and one changed variable.
|
||||
|
||||
## Interpretation Gates
|
||||
|
||||
1. If Host preview or stats presentation slows while Viewer inbound and render
|
||||
remain stable, classify it as local presentation or timer behavior; there is
|
||||
no media defect to repair.
|
||||
2. If JavaScript callback cadence slows while cumulative RTC counters divided by
|
||||
RTCStats timestamp deltas remain stable, classify the observation as sampler
|
||||
cadence, not capture or sender degradation.
|
||||
3. If `RTCVideoSourceStats` frame progress falls, or the track becomes muted,
|
||||
and outbound encoding follows it, localize the issue to the selected source,
|
||||
operating system, or capture backend. A captured source that becomes
|
||||
inaccessible is a platform lifecycle boundary, not evidence for Host-page
|
||||
keepalive.
|
||||
4. If source progress remains stable while outbound encoded progress falls and
|
||||
encode work or `qualityLimitationReason=cpu` agrees, investigate encoder and
|
||||
system load. `qualityLimitationReason` is corroborating evidence, not a sole
|
||||
root-cause oracle.
|
||||
5. If source progress remains stable while outbound rate falls with
|
||||
`qualityLimitationReason=bandwidth`, target/available bitrate, loss, RTT, or
|
||||
retransmission changes, investigate congestion control and the actual path.
|
||||
6. If Host outbound remains stable while Viewer inbound or decode falls,
|
||||
investigate transport or receiver behavior. If inbound/decode is stable but
|
||||
only rendering changes, investigate Viewer presentation lifecycle.
|
||||
7. If capture, outbound, inbound, and decoded progress remain stable but the
|
||||
picture looks blurred, investigate actual codec/profile, bitrate,
|
||||
quantization, and display scaling; do not classify it as background capture
|
||||
throttling.
|
||||
8. If required identities, overlapping windows, or optional metrics are
|
||||
missing, the result is inconclusive. It does not authorize a workaround.
|
||||
|
||||
## Privacy And Evidence Handling
|
||||
|
||||
Raw `chrome://webrtc-internals` dumps stay on the controlled test machine and
|
||||
must not be committed, uploaded, attached to a room, or sent through Screener.
|
||||
They can include page URLs/origins, RTC configuration, candidate addresses,
|
||||
network details, and other fingerprinting material. Do not enable packet,
|
||||
audio-debug, event-log, or media recording for this experiment.
|
||||
|
||||
Only a manually reviewed derived table may become repository evidence. It must
|
||||
replace room, user, session, connection, track, SSRC, and stats IDs with
|
||||
run-local ordinals and omit URLs, origins, hostnames, IP addresses, candidate
|
||||
addresses, device labels/IDs, exact hardware serials, and media content. The
|
||||
allowed durable fields are test version/configuration, coarse environment,
|
||||
surface category, document/source lifecycle state, route category, actual
|
||||
codec/encoder category, bounded metric values, and relative timestamps.
|
||||
|
||||
No new diagnostic server upload, push, sampling timer, ring buffer,
|
||||
persistence, raw stats/SDP/candidate collection, or general telemetry mechanism
|
||||
is approved by this research. If the local tools cannot establish an A/B/C
|
||||
overlap, document the exact missing field first; any new diagnostic consumer
|
||||
requires its own bounded design and privacy review.
|
||||
|
||||
## Mechanisms Outside The Accepted Boundary
|
||||
|
||||
The diagnostic does not adopt:
|
||||
|
||||
- `requestAnimationFrame()`, chained timer, Worker, or synthetic activity loops;
|
||||
- silent audio, `AudioContext` oscillators, hidden/looping media, Picture-in-
|
||||
Picture tricks, focus stealing, or synthetic input;
|
||||
- dummy `RTCDataChannel`, `RTCPeerConnection`, WebSocket, signaling ping, or
|
||||
service-worker activity as a liveness proof;
|
||||
- the Web Screen Wake Lock API as a capture-priority control;
|
||||
- product launch flags that disable browser background policies;
|
||||
- visibility-triggered H.264/VP8 switching, forced H.264, extra sender
|
||||
generations, or automatic capture replacement;
|
||||
- server-side media telemetry, raw diagnostic upload, periodic probes, routing
|
||||
changes, quality-driven reparenting, or magic quality thresholds; or
|
||||
- a Native/executable sender fallback for this browser diagnostic milestone.
|
||||
|
||||
The accepted response to a demonstrated standards/browser capability boundary
|
||||
is a clear documented limitation. A workaround is considered only after the
|
||||
same-machine A/B/C evidence identifies a controllable owner and a minimal
|
||||
standard mechanism changes the remote result without creating another media
|
||||
lifecycle.
|
||||
|
||||
## License Boundary
|
||||
|
||||
This document summarizes public specifications and implementation evidence; it
|
||||
copies no source code and adds no dependency. W3C material is covered by the
|
||||
[W3C Document License](https://www.w3.org/copyright/document-license-2023/),
|
||||
Chromium source files carry their BSD-style license, and Chrome for Developers
|
||||
prose is CC BY 4.0 with samples under Apache 2.0. These sources are evidence
|
||||
only; their implementation is not incorporated into Screener.
|
||||
+1
-1
@@ -7,7 +7,7 @@ Only items in **Now** are executable after their stated decision gate. A branch
|
||||
## Now
|
||||
|
||||
1. **Switch screen-audio ceilings live.** Keep Opus fixed and serialize 64/128/256 kbps desired profile updates across every current Host, relay, and SFU audio sender. Each endpoint uses fresh `getParameters()`/`setParameters()` readback, ignores stale completions, retains media plus its prior applied value on failure, and applies the latest desired value to future edges. Do not add a remote applied-ack protocol or claim room-wide atomic convergence in this slice.
|
||||
2. **Switch video codec while paused.** Serialize the change with route mutation, freeze exact current route/media bindings, hold new joins, and boundedly renegotiate P2P/relay plus republish SFU while paused. Preparation must not resume a share the user explicitly paused: after it settles, stop the deadline but retain the single room-serial codec owner as one bounded prepared record with no timer. The user's normal Resume first atomically revalidates those bindings in that owner, starts the decoded-frame proof deadline, and only then enables the source. A binding invalidated by leave or hard route failure exits this proof set and waits for routing to rebuild after the transaction settles. Preparation failure remains paused; proof failure first restores authoritative pause and performs bounded rollback preparation, whose proof waits for the user's next Resume. Rollback proves the prior preference was reapplied plus a newly negotiated actual codec and decoded progress, not literal equality with the old actual codec. No path auto-resumes. Zero targets update only the future-edge preference. Reset `automatic` with empty codec preferences. This is a paused bounded-gap transaction; unpaused make-before-break is a later extension.
|
||||
2. **Switch video codec while paused.** Serialize the change with route mutation, freeze exact current route/media bindings, hold new joins, and boundedly renegotiate P2P/relay plus republish SFU while paused. Preparation must not resume a share the user explicitly paused: after it settles, stop the deadline but retain the single room-serial codec owner as one bounded prepared record with no timer. The user's normal Resume first atomically revalidates those bindings in that owner, allocates the next server-monotonic proof attempt, starts the decoded-frame proof deadline, and only then enables the source. Proof request and response carry that exact attempt; Pause invalidates it, so a delayed proof from an earlier Resume cannot settle a later proof window. A binding invalidated by leave or hard route failure exits this proof set and waits for routing to rebuild after the transaction settles. Preparation failure remains paused; proof failure first restores authoritative pause and performs bounded rollback preparation, whose proof waits for the user's next Resume. Rollback proves the prior preference was reapplied plus a newly negotiated actual codec and decoded progress, not literal equality with the old actual codec. No path auto-resumes. Zero targets update only the future-edge preference. Reset `automatic` with empty codec preferences. This is a paused bounded-gap transaction; unpaused make-before-break is a later extension.
|
||||
3. **Diagnose the dated H.264/blur report.** Reproduce the 2026-08-19 then-production release `769de201f7cc` against current `main` on the same Host/browser/driver/game and one wired direct Viewer, changing one variable at a time. Implement a codec-specific repair only if codec/encoder evidence identifies it; otherwise fix the proven capture, sender, BWE, or lifecycle cause.
|
||||
4. **Diagnose desktop Host background/minimized performance.** On the same machine, dynamic scene, direct Viewer, browser, and fresh share, compare foreground, occluded, minimized, and hidden states while separately verifying actual negotiated VP8/H.264. Correlate capture settings/FPS, frames captured/encoded/sent/decoded, bitrate, `qualityLimitationReason`, encode time, browser/game CPU and GPU, and the existing local-preview pause boundary. Treat browser scheduling/throttling and codec effects as hypotheses until measured. Do not spoof page activity, add keepalive timers, or misuse Wake Lock; if current Chrome/Edge cannot preserve the accepted Host behavior through standard Web APIs, record that capability boundary before considering the packaged/native Host.
|
||||
5. **Remove the unused parent-edge quality-proof surface.** Delete the client reporter, protocol message, server dispatch/ack flag, benchmark sender, and tests whose only purpose is the unconsumed `parent-edge-quality-evidence` message. Keep the existing exact Viewer evidence generation/session gate used by diagnostics. Do not replace the deleted magic packet/loss thresholds with another score, probe, blacklist, or route input.
|
||||
|
||||
+1
-1
@@ -160,7 +160,7 @@ Zod 在服务边界拒绝未知类型、超长字符串、非法 SDP/candidate
|
||||
|
||||
每次 `setParameters()` 都必须从当次 `getParameters()` 派生更新，然后读回 requested/applied `maxBitrate`、`maxFramerate`、`scaleResolutionDownBy` 和 degradation preference。浏览器负责具体降级，实际结果由 stats 观测；拒绝或浏览器改写必须进入可见状态，不能只写 `console.warn`。
|
||||
|
||||
Web 视频端不做视频 SDP munging。`automatic` 以空 `setCodecPreferences([])` 恢复 P2P 浏览器默认且不覆盖 LiveKit 默认 codec；显式 H.264/VP8 在首个 offer/publication 前应用，P2P 与 browser relay 仍保留完整 browser codec/repair 回退列表，SFU 始终保持 `backupCodec: false`。暂停时的 codec 切换使用单调 codec-negotiation generation，并和 route mutation 串行：冻结 exact current route/media binding target，新 join 按 pause 语义等待、leave 从 target 移除；P2P/relay 在暂停中以独立总 deadline 显式重新 offer/answer，SFU 以新 publication generation republish。Preparation 成功后只停止 preparation deadline；同一个 room-serial codec mutation owner 保留一个内存有界的 prepared generation，暂停期间无 timer并排斥 route mutation，绝不擅自恢复用户主动暂停的分享。Host 的正常 Resume 动作先在该 owner 内原子重验 frozen bindings，启动 proof deadline 后才启用 source。每名 still-current target 从实际 inbound RTP stats 与 decoded-frame counter 生成专用 codec-transition proof；proof 精确绑定 room/share、Viewer session、assignment/connection/media-binding/publication generation 与 codec-negotiation generation，不复用 Viewer quality evidence，也不改变 route authority。leave、hard failure 或 binding generation 变化会移除该 target并把仍在线 Viewer置为 `route-wait`；事务以剩余 target settle 后释放 owner，controller 再使用最终 committed preference 恢复该 Viewer。服务端只在全部 still-current target 的 actual codec 与 decoded progress 成立后提交 desired codec。Preparation failure 保持暂停并运行有界 old-preference rollback preparation；forward proof failure 先重新进入权威 paused，再运行 rollback preparation。Rollback prepared state 保留同一 owner且无 timer，等待用户下一次正常 Resume 后重验、启动 rollback proof deadline并启用 source；proof 证明 P2P 空/有序 preference 或 SFU fresh publish options 已恢复，并从新协商代次取得 actual inbound codec 与 decoded progress，不要求 actual codec 与旧观测字面相同。回滚失败保持暂停和明确错误，任何路径都不自动 Resume。stop/share replacement/room delete 清除 prepared state；零 target 只更新 future-edge desired codec。该 slice 是 paused bounded-gap mutation，不声称 make-before-break；`replaceTrack()` 或暂停本身不改变 codec，未暂停热切换后续复用同一事务。实际 outbound codec/profile、encoder implementation 与 stats 是结果真相；浏览器支持 H.264 不证明命中硬编。AV1 仍不得仅凭压缩率推断默认启用。唯一获准的 Web SDP 变换是下述 Viewer answer 的 Opus stereo receive preference，不得借此扩展视频码率、codec 或通用 munging。
|
||||
Web 视频端不做视频 SDP munging。`automatic` 以空 `setCodecPreferences([])` 恢复 P2P 浏览器默认且不覆盖 LiveKit 默认 codec；显式 H.264/VP8 在首个 offer/publication 前应用，P2P 与 browser relay 仍保留完整 browser codec/repair 回退列表，SFU 始终保持 `backupCodec: false`。暂停时的 codec 切换使用单调 codec-negotiation generation，并和 route mutation 串行：冻结 exact current route/media binding target，新 join 按 pause 语义等待、leave 从 target 移除；P2P/relay 在暂停中以独立总 deadline 显式重新 offer/answer，SFU 以新 publication generation republish。Preparation 成功后只停止 preparation deadline；同一个 room-serial codec mutation owner 保留一个内存有界的 prepared generation，暂停期间无 timer并排斥 route mutation，绝不擅自恢复用户主动暂停的分享。Host 的正常 Resume 动作先在该 owner 内原子重验 frozen bindings，由服务端为本次证明窗口分配该事务内严格递增的 `proofAttempt`，启动 proof deadline 后才启用 source。每名 still-current target 从实际 inbound RTP stats 与 decoded-frame counter 生成专用 codec-transition proof；proof 精确绑定 room/share、Viewer session、assignment/connection/media-binding/publication generation、codec-negotiation generation 与当前 `proofAttempt`，request 和 response 必须回显同一 attempt。Pause 清空当前 proof batch并使该 attempt 失效；下一次 Resume 分配新 attempt，因此任何较早窗口延迟到达的 proof 都不能提交当前窗口。不复用 Viewer quality evidence，也不改变 route authority。leave、hard failure 或 binding generation 变化会移除该 target并把仍在线 Viewer置为 `route-wait`；事务以剩余 target settle 后释放 owner，controller 再使用最终 committed preference 恢复该 Viewer。服务端只在全部 still-current target 的 actual codec 与 decoded progress 成立后提交 desired codec。Preparation failure 保持暂停并运行有界 old-preference rollback preparation；forward proof failure 先重新进入权威 paused，再运行 rollback preparation。Rollback prepared state 保留同一 owner且无 timer，等待用户下一次正常 Resume 后重验、分配新 proof attempt、启动 rollback proof deadline并启用 source；proof 证明 P2P 空/有序 preference 或 SFU fresh publish options 已恢复，并从新协商代次取得 actual inbound codec 与 decoded progress，不要求 actual codec 与旧观测字面相同。回滚失败保持暂停和明确错误，任何路径都不自动 Resume。stop/share replacement/room delete 清除 prepared state；零 target 只更新 future-edge desired codec。该 slice 是 paused bounded-gap mutation，不声称 make-before-break；`replaceTrack()` 或暂停本身不改变 codec，未暂停热切换后续复用同一事务。实际 outbound codec/profile、encoder implementation 与 stats 是结果真相；浏览器支持 H.264 不证明命中硬编。AV1 仍不得仅凭压缩率推断默认启用。唯一获准的 Web SDP 变换是下述 Viewer answer 的 Opus stereo receive preference，不得借此扩展视频码率、codec 或通用 munging。
|
||||
|
||||
Web display-audio 请求显式关闭 `echoCancellation`、`noiseSuppression`、`autoGainControl` 和 `voiceIsolation`，并以 `channelCount: { ideal: 2 }` 偏好双声道；这是当前 Chrome/Edge Host 基线的实现契约，用于避开网页 `getDisplayMedia()` 默认的语音处理和单声道 APM 候选，不宣称跨浏览器保证，也不固定采样率。所有值都是非强制 post-selection preference；不使用 `exact`、`min` 或 `advanced`。若 `getAudioTracks()` 为空，分享者界面必须显示“当前来源没有可共享音频”。Web 端同时传入 `windowAudio: "window"` 与 `systemAudio: "include"`：窗口选项请求该窗口/应用音频，整个屏幕选项允许用户勾选系统音频；标签页使用浏览器自身的标签页音频选择。浏览器可以忽略这些约束或 picker hints，且没有对应的标准来源类型读回，因此 UI 不额外推断或标注来源。系统音频由用户显式选择，可能包含语音或通知；video-only 始终可用。
|
||||
|
||||
|
||||
+1
-1
@@ -168,7 +168,7 @@ SFU 的 publication、subscription、server-resource accounting、临时 overlap
|
||||
- Given Web Host 或浏览器 relay 建立或替换分享音轨，When 对实际音频 sender 应用发送参数，Then 从当次 `getParameters()` 派生所选 `encodings[0].maxBitrate = 64000 | 128000 | 256000`、调用 `setParameters()` 并立即读回，旧配置缺失时按 128000 处理；Given Web Viewer 为含唯一有效 audio/Opus payload 的 offer 生成 answer，Then 用结构化 SDP parser 只 upsert 该 payload 的 `stereo=1` 与 `maxaveragebitrate=256000`，并把同一个结果用于 `setLocalDescription()` 和 signaling。首次连接、ICE restart、同 parent rebuild 和 relay child 都适用；空 SDP、无/歧义 Opus、多个有效 audio、重复目标参数、畸形 fmtp 或 parse/write 异常必须按对象 identity 返回浏览器原 answer 并继续，不能发送部分修改或静默冒充 stereo/实际码率。只有实际 `setLocalDescription()` 失败才进入既有连接错误路径。`sprop-stereo` 与 `usedtx` 不新增，其他 fmtp/FEC/session/media 字段逐项保留。
|
||||
- Given 通过 SFU 发布分享音轨，When 构建 LiveKit 2.22.0 publish options，Then 64 kbps 使用 `AudioPresets.musicStereo`、128 kbps 使用 `AudioPresets.musicHighQualityStereo`、256 kbps 使用同等结构的显式 sender ceiling，三档都设置 `forceStereo = true` 与 `dtx = false`，且不覆盖 pinned room default `red = true`；验收同时核对 SDK 选项、applied ceiling、当前 RTP codec/fmtp 与 stats 实际码率，不能从 preset 名称或 `opus/48000/2` 单独推断实际声道内容。
|
||||
- Given 分享正在直播或暂停且 Host 选择新的 64/128/256 kbps 音频档位，When 当前 profile mutation 串行执行，Then 服务端以 last-wins 保存并广播最新 desired profile，每个当前 Host、relay 与 SFU audio sender 从 fresh `getParameters()` 派生 `setParameters()` 并立即 readback，过期 completion 不得覆盖新值；以后创建的 sender 使用最新 desired profile。单个端点应用失败时保留媒体与其旧 applied 值、显示可见告警并允许后续更新或重建重试，不得谎称全房已收敛。当前 applied readback 保持 endpoint-local；没有显式回执 wire 就不宣称跨端点原子提交。该操作不改变 Opus、route revision、media-binding generation 或 publication generation，也不创建第二个音频控制环。当前 screen-audio Host 基线是 Chrome/Edge；其他浏览器只有实际 sender readback 成功时才算支持。
|
||||
- Given 分享已显式暂停且 Host 选择 `自动 | H.264 | VP8` 的不同值，When 执行首个视频 codec 切换闭环，Then 为该请求分配单调 codec-negotiation generation，并与 room route mutation 串行；事务开始时冻结 exact current route/media binding target，离开的 Viewer 从 target 移除，新加入者按既有 pause 语义等待。暂停期间在独立有界 preparation deadline 内对 target 的 P2P/relay path 完成显式 offer/answer 重协商，并对 SFU 路径完成新的 publication generation/republish；选择 `automatic` 时以空 codec preference 恢复浏览器默认。Preparation 完成不得擅自恢复用户主动暂停的分享：停止 preparation deadline，但继续由同一个 room-serial codec mutation owner 持有一个 bounded prepared generation，暂停期间不运行 timer并排斥 route mutation。Host 的正常 Resume 必须先在该 owner 内原子重验全部 frozen binding；仍精确匹配时启动 proof deadline，然后才启用 source。暂停帧不能充当媒体证明：用户恢复 source 后，每名 still-current target 发送一个只属于 codec transaction 的 proof，精确绑定 room/share、Viewer session、assignment/connection/media-binding/publication generation、codec-negotiation generation、实际 inbound codec 与 decoded-frame baseline/delta；该 proof 不进入路由失效判断。若 leave、hard failure 或任何 exact binding 变化使 target 失效，服务端把它移出本事务并标为 `route-wait`，codec 事务以剩余 target settle 后释放 owner，controller 再用最终 committed preference 为仍在线 Viewer 恢复路径。只有服务端校验全部 still-current proof 后才提交 desired codec。Preparation failure 保持暂停并开启有界 old-preference rollback preparation；forward proof failure 则先重新进入服务端权威 paused，再执行同一 rollback preparation。Rollback prepared state继续占有同一 owner、无 timer，并等待用户下一次正常 Resume 后重验 binding、启动 rollback proof deadline、再启用 source；rollback 证明旧 preference 已重新应用（P2P 空/有序 preference 或 SFU fresh publish options）以及新协商代次的 actual inbound codec 与 decoded progress，不要求 actual codec 字面等于历史值。回滚失败保持暂停并显示明确失败；任何路径都不自动 Resume，不允许 stale generation 恢复播放。停止分享、替换 share generation 或删除房间清除 prepared state。零 current Viewer 或事务中全部离开时直接提交 future-edge desired codec，无需伪造 frame proof。`pause`、`replaceTrack()` 或仅调用 `setCodecPreferences()` 均不能冒充已完成切换；本 slice 是明确的 paused bounded-gap 事务，未暂停 MBB 热切换留作后续扩展。
|
||||
- Given 分享已显式暂停且 Host 选择 `自动 | H.264 | VP8` 的不同值，When 执行首个视频 codec 切换闭环，Then 为该请求分配单调 codec-negotiation generation，并与 room route mutation 串行；事务开始时冻结 exact current route/media binding target，离开的 Viewer 从 target 移除，新加入者按既有 pause 语义等待。暂停期间在独立有界 preparation deadline 内对 target 的 P2P/relay path 完成显式 offer/answer 重协商，并对 SFU 路径完成新的 publication generation/republish；选择 `automatic` 时以空 codec preference 恢复浏览器默认。Preparation 完成不得擅自恢复用户主动暂停的分享：停止 preparation deadline，但继续由同一个 room-serial codec mutation owner 持有一个 bounded prepared generation，暂停期间不运行 timer并排斥 route mutation。Host 的正常 Resume 必须先在该 owner 内原子重验全部 frozen binding，由服务端分配该事务内严格递增的 `proofAttempt`，启动 proof deadline，然后才启用 source。暂停帧不能充当媒体证明：用户恢复 source 后，每名 still-current target 发送一个只属于 codec transaction 的 proof，精确绑定 room/share、Viewer session、assignment/connection/media-binding/publication generation、codec-negotiation generation、当前 `proofAttempt`、实际 inbound codec 与 decoded-frame baseline/delta；request 和 response 必须回显同一 attempt，Pause 立即使当前 attempt 失效，因此旧 Resume 窗口的延迟 proof 不能提交新窗口。该 proof 不进入路由失效判断。若 leave、hard failure 或任何 exact binding 变化使 target 失效，服务端把它移出本事务并标为 `route-wait`，codec 事务以剩余 target settle 后释放 owner，controller 再用最终 committed preference 为仍在线 Viewer 恢复路径。只有服务端校验全部 still-current proof 后才提交 desired codec。Preparation failure 保持暂停并开启有界 old-preference rollback preparation；forward proof failure 则先重新进入服务端权威 paused，再执行同一 rollback preparation。Rollback prepared state继续占有同一 owner、无 timer，并等待用户下一次正常 Resume 后重验 binding、分配新 proof attempt、启动 rollback proof deadline、再启用 source；rollback 证明旧 preference 已重新应用（P2P 空/有序 preference 或 SFU fresh publish options）以及新协商代次的 actual inbound codec 与 decoded progress，不要求 actual codec 字面等于历史值。回滚失败保持暂停并显示明确失败；任何路径都不自动 Resume，不允许 stale generation 或 stale proof attempt 恢复播放。停止分享、替换 share generation 或删除房间清除 prepared state。零 current Viewer 或事务中全部离开时直接提交 future-edge desired codec，无需伪造 frame proof。`pause`、`replaceTrack()` 或仅调用 `setCodecPreferences()` 均不能冒充已完成切换；本 slice 是明确的 paused bounded-gap 事务，未暂停 MBB 热切换留作后续扩展。
|
||||
- Given 浏览器返回屏幕音轨，When 产品展示音频状态或未来连接详情，Then 只能把存在性、浏览器实际返回的 track settings、实际协商 codec/fmtp 和实际流量分别标为捕获、协商或观测事实；字段缺失必须显示为未知，且不得从 Opus 的 `48000/2` RTP 声明推断源采样率或实际 stereo。
|
||||
- Given Web 或后续 native sender 分享电影/视频音频，When 进行产品验收，Then 必须用独立 A/B/C 证据关联 capture audio track/settings、实际 negotiated codec/fmtp、outbound/inbound 实际码率、loss、jitter、concealment、jitter-buffer 与 A/V sync；P2P、浏览器 relay 与 SFU route switch 保持同一持久 Viewer `MediaStream`/媒体元素和 stereo 协商，不新增独立 audio element 或 Web Audio 时钟边界。native sender 必须通过该音质和同步矩阵，且不得把 audio `maxBitrate` ceiling 当作提质证明。
|
||||
- Given 分享者同时运行语音软件，When 选择仅分享游戏声音，Then supported Windows 11 native sender 只保留目标游戏及其子进程的渲染音频，观众、通知和其他应用声音不得进入分享；浏览器 system audio 无法满足时必须允许 video-only 或明确告知其可能包含语音/通知，不能用 AEC、降噪或来源标签伪称已隔离。
|
||||
|
||||
Reference in new issue
Block a user