fix(media): stabilize sharing startup and recovery

## Problem and outcome / 问题与结果

Sharing could fail when an optional output or encoder comparison failed, while retired Browser work could block media cleanup. Browser pooling also spent its startup protection before publication and classified its synthetic transport clock as realtime video, delaying recovery after a bandwidth drop. These repairs stay within the existing capture, producer, edge and connection owners.

Self-hosting samples now follow the configured public Origin and permit the site's requested camera/microphone capture. Piik's own Origin rejection has a specific bilingual explanation, distinct from arbitrary proxy 403 responses.

## Release notes

### 中文

这是一次分享稳定性维护更新,重点修复开播、换源与画质恢复,并改善自部署站点的配置和错误说明。

- Windows 自动选择编码器时,如果已验证 H.264 可用,后续软件编码对比失败或超时会保留可用的 H.264,避免因此无法开播。
- 较低画质输出或单条音频发送连接失败时,保留仍然健康的分享和其他连接。
- 修复停止分享、换源和媒体连接退役时,旧异步任务阻塞清理与恢复的问题。
- 改善浏览器共享编码的首播画质与网络波动后的恢复,避免因短时码率变化反复重建编码器。
- 自部署配置示例默认跟随站点网址;创建房间因网址未获允许而返回 403 时,页面会指出需要检查的配置。nginx 示例也修正了对本站摄像头、麦克风的权限限制。

已有自部署站点若遇到 403,请检查 `PUBLIC_BASE_URL` 与 `ALLOWED_ORIGINS`。使用旧 nginx 示例的站点需更新 `Permissions-Policy`;步骤见[自部署教程](https://piik.tv/docs/zh/self-hosting)。本次更新不改变房间、信令或 App 控制协议。

### English

This maintenance update improves sharing startup, source changes and video-quality recovery, with clearer configuration and error guidance for self-hosted sites.

- Windows automatic codec selection retains a verified H.264 encoder if the subsequent software comparison fails or times out, instead of failing to start sharing.
- A failed lower-quality output or individual audio sender no longer interrupts healthy sharing and other connections.
- Fixes retired asynchronous work blocking cleanup and recovery when stopping sharing, changing sources or retiring media connections.
- Improves initial quality and recovery after network fluctuations in Browser pooled encoding, while avoiding unnecessary encoder recreation after short bitrate changes.
- The self-hosting sample follows the configured site address by default. Piik's Origin-related room-creation 403 now identifies the settings to check. The nginx sample also permits camera and microphone requests from the site itself.

If an existing self-hosted site returns 403, check `PUBLIC_BASE_URL` and `ALLOWED_ORIGINS`. Sites using the old nginx sample must update `Permissions-Policy`; see the [self-hosting guide](https://piik.tv/docs/self-hosting). Room, signaling and App control protocols are unchanged.

## Verification / 验证

- `npm run check`: 75 files / 1,106 tests, TypeScript and production Web build passed.
- `npm run check:go`: Go formatting, tests, vet, supported core builds, Windows native compile and regressions passed. Darwin cgo packaging remains owned by its macOS CI runner.
- Repository hygiene, documentation checks and website/documentation build passed.
- Isolated VP8/H.264 Browser checks: constrained-child isolation and recovery, short congestion pulse, live and quiet settings, pause/resume, source replacement and ordinary fallback passed. Controlled bandwidth can still cause temporary low resolution, stalls and A/V jitter.
- Windows native Auto/H.264/VP8 startup, window resize/fullscreen/minimize/restore, Browser-to-App fanout and embedded-SFU checks passed in the bounded local fixtures.
- Old-code negative controls reproduce the repaired failure paths. A real Chromium policy check rejects camera/microphone with the previous nginx header and captures both fake devices with the corrected same-origin policy; geolocation remains denied.
- Independent read-only review covered root ownership, cancellation and the producer/carrier adaptation boundary.

## Impact / 影响

Compatible patch release, planned as v1.6.4. No wire, storage, routing-authority or interaction-model change. Matching Server restart interrupts current process-owned media; package cutover retains the existing bounded rollback procedure. Unknown field reports still require device/network evidence and are not claimed resolved by local fixtures. Raw diagnostics and experiment artifacts remain ignored; accepted constraints and concise evidence live in their existing documentation owners.
This commit is contained in:
AAA authored and GitHub committed 2026-09-25 14:42:59 +08:00
1 parent f423df0acf
commit 383f3656c7
50 files changed
+1571 -263

No files matched your search

+3 -2
View File
@@ -4,8 +4,9 @@ LISTEN_HOST=0.0.0.0
PORT=8787 PORT=8787
# For LAN viewers, replace localhost with this computer's LAN address. # For LAN viewers, replace localhost with this computer's LAN address.
PUBLIC_BASE_URL=http://localhost:8787 PUBLIC_BASE_URL=http://localhost:8787
# Include both the host browser origin and PUBLIC_BASE_URL when they differ. # Leave empty to follow PUBLIC_BASE_URL. For multiple trusted browser addresses,
ALLOWED_ORIGINS=http://localhost:8787 # list all exact origins (scheme, host and port), including PUBLIC_BASE_URL.
ALLOWED_ORIGINS=
# Optional site access password in every environment. Unset or empty allows # Optional site access password in every environment. Unset or empty allows
# entry without a site password; room ownership and invitations still apply. # entry without a site password; room ownership and invitations still apply.
# Non-empty values are matched exactly; spaces and Unicode are supported. # Non-empty values are matched exactly; spaces and Unicode are supported.
+1 -1
View File
@@ -37,7 +37,7 @@ server {
add_header X-Content-Type-Options "nosniff" always; add_header X-Content-Type-Options "nosniff" always;
add_header Referrer-Policy "no-referrer" always; add_header Referrer-Policy "no-referrer" always;
add_header X-Frame-Options "DENY" always; add_header X-Frame-Options "DENY" always;
add_header Permissions-Policy "camera=(), microphone=(), geolocation=()" always; add_header Permissions-Policy "camera=(self), microphone=(self), geolocation=()" always;
# Media uses /signal. Reject credential-bearing RTC query endpoints without # Media uses /signal. Reject credential-bearing RTC query endpoints without
# persisting their query strings, including requests from already-open pages. # persisting their query strings, including requests from already-open pages.
+2 -1
View File
@@ -98,7 +98,8 @@ and [website preview and publishing](./operations/website.md).
| Host and viewer behavior | [Presentation and lifecycle](./standards/presentation-lifecycle.md) | | Host and viewer behavior | [Presentation and lifecycle](./standards/presentation-lifecycle.md) |
| Why a design was chosen | [Architecture decisions](./adr/) | | Why a design was chosen | [Architecture decisions](./adr/) |
| Measurements and platform limits | [Research](./research/) and [verification status](./verification-status.md) | | Measurements and platform limits | [Research](./research/) and [verification status](./verification-status.md) |
| Camera and Host microphone candidate | [Capture scope and limits](./research/camera-and-microphone.md) | | Camera and Host microphone | [Capture scope and limits](./research/camera-and-microphone.md) |
| Room voice proposal and media-fidelity boundaries | [Design assessment](./research/room-voice.md) |
## Shared conventions ## Shared conventions
+19
View File
@@ -16,6 +16,7 @@ collect a diagnostic report if the problem persists.
| Yellow outline around the shared window or screen | This is Windows' capture indicator. See [capture borders](#yellow-capture-border-on-windows) for Windows 11 controls and an optional Windows 10 workaround. | | Yellow outline around the shared window or screen | This is Windows' capture indicator. See [capture borders](#yellow-capture-border-on-windows) for Windows 11 controls and an optional Windows 10 workaround. |
| App startup fails | Read the reason on the page and in the terminal. Reopen the App and enable the chip-shaped **Debug launch** control after the theme button before trying again. A failed startup then exports a report; its path appears in the terminal. | | App startup fails | Read the reason on the page and in the terminal. Reopen the App and enable the chip-shaped **Debug launch** control after the theme button before trying again. A failed startup then exports a report; its path appears in the terminal. |
| Local invitation will not open | Check that both devices are on the same network and can reach each other. Guest Wi-Fi or firewall rules can block local access. | | Local invitation will not open | Check that both devices are on the same network and can reach each other. Guest Wi-Fi or firewall rules can block local access. |
| Cannot create a room (403) | On a self-hosted site, ask the administrator to check the [allowed site address](#room-creation-returns-403). |
| Page opens but video will not connect | Use **Reconnect** if available, or refresh the viewing page. See [connection troubleshooting](#when-video-will-not-connect) if it still fails. | | Page opens but video will not connect | Use **Reconnect** if available, or refresh the viewing page. See [connection troubleshooting](#when-video-will-not-connect) if it still fails. |
| Sharing stops after sleep or suspension | Wake the device and return to the sharing tab; start sharing again if needed. Browser and OS suspension can interrupt capture or playback. | | Sharing stops after sleep or suspension | Wake the device and return to the sharing tab; start sharing again if needed. Browser and OS suspension can interrupt capture or playback. |
@@ -23,6 +24,24 @@ For a bug report, include the version, OS/browser, what you expected, and how
to reproduce it. [Diagnostics and export](../../cmd/piik-app/README.md#diagnostics) to reproduce it. [Diagnostics and export](../../cmd/piik-app/README.md#diagnostics)
explains how to collect a local report and what to review before sharing it. explains how to collect a local report and what to review before sharing it.
## Room creation returns 403
Piik rejects room creation from a browser address that the server has not allowed,
even when the page opens normally. The site administrator should:
1. Set `PUBLIC_BASE_URL` to the browser's site address, including the scheme and
any non-default port, without a path.
2. Unset or empty `ALLOWED_ORIGINS` to use that address. If several addresses are
needed, list every trusted origin explicitly, separated by commas. Check for
an old `http://localhost:8787` override after copying a configuration example.
3. Restart Piik (recreate the container with `docker compose up -d` for Compose),
reload the page and retry.
If the response says `Origin not allowed` with correct settings, check that the
reverse proxy preserves the browser's `Origin` header. A 403 from the proxy or
WAF itself needs its own logs; do not disable origin validation to bypass it.
See [Server deployment](../operations/self-hosting.md) for the configuration.
## App windows or screens are missing ## App windows or screens are missing
Keep Piik App running on the computer where you are sharing. Open the current Keep Piik App running on the computer where you are sharing. Open the current
+15
View File
@@ -16,6 +16,7 @@
| 分享的窗口或屏幕周围出现黄框 | 这是 Windows 的采集提示。查看[采集边框说明](#windows-分享时出现黄框),了解 Windows 11 的开关和 Windows 10 的可选处理方法。 | | 分享的窗口或屏幕周围出现黄框 | 这是 Windows 的采集提示。查看[采集边框说明](#windows-分享时出现黄框),了解 Windows 11 的开关和 Windows 10 的可选处理方法。 |
| App 启动失败 | 先查看页面和终端中的具体原因。重新打开 App,在主题按钮后的芯片图标开启「诊断启动」再重试。启动失败时会自动导出报告,保存位置见终端。 | | App 启动失败 | 先查看页面和终端中的具体原因。重新打开 App,在主题按钮后的芯片图标开启「诊断启动」再重试。启动失败时会自动导出报告,保存位置见终端。 |
| 本地邀请打不开 | 确认两台设备在同一网络且可以互相访问。访客 Wi-Fi 或防火墙规则可能阻止本地连接。 | | 本地邀请打不开 | 确认两台设备在同一网络且可以互相访问。访客 Wi-Fi 或防火墙规则可能阻止本地连接。 |
| 无法创建房间 (403) | 自建站点请联系管理员检查[允许访问的站点地址](#创建房间返回-403)。 |
| 网页能打开,画面连不上 | **重新连接** 按钮可用时先点击,否则刷新观看页。仍然失败时,查看 [画面连接排查](#画面连接不上)。 | | 网页能打开,画面连不上 | **重新连接** 按钮可用时先点击,否则刷新观看页。仍然失败时,查看 [画面连接排查](#画面连接不上)。 |
| 休眠或挂起后分享中断 | 唤醒设备,回到分享标签页,必要时重新开始分享。浏览器或系统挂起可能中断采集和播放。 | | 休眠或挂起后分享中断 | 唤醒设备,回到分享标签页,必要时重新开始分享。浏览器或系统挂起可能中断采集和播放。 |
@@ -23,6 +24,20 @@
[诊断与导出](../../cmd/piik-app/README.zh-CN.md#诊断) [诊断与导出](../../cmd/piik-app/README.zh-CN.md#诊断)
说明如何收集本地报告,以及分享前需要检查哪些内容。 说明如何收集本地报告,以及分享前需要检查哪些内容。
## 创建房间返回 403
浏览器访问的网址不在服务端允许的范围内时,页面可能正常打开,但创建房间会被拒绝。
站点管理员可以按以下步骤检查:
1. 将 `PUBLIC_BASE_URL` 设为浏览器访问的站点地址,包含协议和非默认端口,不带路径。
2. `ALLOWED_ORIGINS` 不设置或留空时使用上述地址;需要多个地址时,用英文逗号列出全部
可信地址。复制过配置示例的话,检查这里是否仍残留 `http://localhost:8787`。
3. 重启 Piik;使用 Compose 时运行 `docker compose up -d` 重建容器。刷新网页后重试。
配置正确但响应仍是 `Origin not allowed` 时,检查反向代理是否保留浏览器的 `Origin`
请求头。如果 403 来自代理或 WAF 本身,则应查看对应日志,不要通过关闭来源校验来绕过。
配置方法见 [Server 部署指南](../operations/self-hosting.zh-CN.md)。
## App 找不到窗口或屏幕 ## App 找不到窗口或屏幕
在分享画面的这台电脑上保持 Piik App 运行,并从 App 打开当前站点。 在分享画面的这台电脑上保持 Piik App 运行,并从 App 打开当前站点。
+11
View File
@@ -76,6 +76,11 @@ working directory. A blank `SITE_ACCESS_PASSWORD` allows
entry without a site passphrase; enter a passphrase to require one. entry without a site passphrase; enter a passphrase to require one.
Room invitations and access settings still apply. Room invitations and access settings still apply.
`PUBLIC_BASE_URL` must match the address opened in the browser, including HTTPS
and any non-default port. Leave `ALLOWED_ORIGINS` unset or empty for this single
address. An existing non-empty value overrides that default; a stale value can
cause [403 when creating a room](../guide/troubleshooting.md#room-creation-returns-403).
`MAX_VIEWERS_PER_ROOM` sets the room's Viewer limit, excluding the Host. Choose `MAX_VIEWERS_PER_ROOM` sets the room's Viewer limit, excluding the Host. Choose
`1..20` and restart to apply it. More Viewers can require more network and relay `1..20` and restart to apply it. More Viewers can require more network and relay
resources. See [room capacity](../standards/configuration.md#room-capacity) resources. See [room capacity](../standards/configuration.md#room-capacity)
@@ -96,6 +101,12 @@ share.example.com {
Reload Caddy. It obtains and renews the certificate automatically when DNS points Reload Caddy. It obtains and renews the certificate automatically when DNS points
to the server and TCP 80/443 are reachable. See [Caddy's HTTPS proxy guide](https://caddyserver.com/docs/quick-starts/reverse-proxy#https). to the server and TCP 80/443 are reachable. See [Caddy's HTTPS proxy guide](https://caddyserver.com/docs/quick-starts/reverse-proxy#https).
For nginx, see the [configuration example](../../deploy/nginx/piik.conf.example).
If you copied an older example, change its `Permissions-Policy` to
`camera=(self), microphone=(self), geolocation=()` and reload nginx. Empty
camera/microphone allowlists prevent Browser capture even when the user grants
permission; `self` allows this site to request access.
### 3. Open the ports and verify ### 3. Open the ports and verify
Allow **TCP 80/443** for HTTPS and **UDP 3478** for STUN in the server firewall Allow **TCP 80/443** for HTTPS and **UDP 3478** for STUN in the server firewall
+8
View File
@@ -71,6 +71,10 @@ SITE_ACCESS_PASSWORD=
`SITE_ACCESS_PASSWORD` 留空时,进入站点无须口令; `SITE_ACCESS_PASSWORD` 留空时,进入站点无须口令;
如需设置口令,填写你想使用的口令即可。房间邀请与加入权限仍由房主管理。 如需设置口令,填写你想使用的口令即可。房间邀请与加入权限仍由房主管理。
`PUBLIC_BASE_URL` 应与浏览器访问的网址一致,包括 HTTPS 和非默认端口。
只使用这个地址时,`ALLOWED_ORIGINS` 不设置或留空即可;已有的非空值会覆盖此默认值,
旧地址未更新可能导致[创建房间返回 403](../guide/troubleshooting.zh-CN.md#创建房间返回-403)。
`MAX_VIEWERS_PER_ROOM` 可设置每房观众上限,不含房主,支持 `1..20`,修改后重启生效。 `MAX_VIEWERS_PER_ROOM` 可设置每房观众上限,不含房主,支持 `1..20`,修改后重启生效。
人数越多,对网络和转发资源的需求也可能增加。默认值及 App 房间的区别见 人数越多,对网络和转发资源的需求也可能增加。默认值及 App 房间的区别见
[人数限制](../standards/configuration.md#room-capacity)。 [人数限制](../standards/configuration.md#room-capacity)。
@@ -89,6 +93,10 @@ share.example.com {
重新加载 Caddy。域名解析正确且 TCP 80/443 可访问时,它会自动申请和续期证书。 重新加载 Caddy。域名解析正确且 TCP 80/443 可访问时,它会自动申请和续期证书。
具体操作见 [Caddy 的 HTTPS 代理说明](https://caddyserver.com/docs/quick-starts/reverse-proxy#https)。 具体操作见 [Caddy 的 HTTPS 代理说明](https://caddyserver.com/docs/quick-starts/reverse-proxy#https)。
nginx 可参考[配置示例](../../deploy/nginx/piik.conf.example)。如果沿用了旧示例,请将
`Permissions-Policy` 改为 `camera=(self), microphone=(self), geolocation=()`,然后重新加载 nginx。
摄像头和麦克风的空白允许列表会阻止浏览器采集,即使用户已授权;`self` 允许本站请求访问。
### 3. 放行端口并检查 ### 3. 放行端口并检查
在服务器防火墙和云平台安全组中放行 **TCP 80/443**(HTTPS)和 **UDP 3478**(STUN)。 在服务器防火墙和云平台安全组中放行 **TCP 80/443**(HTTPS)和 **UDP 3478**(STUN)。
@@ -93,6 +93,64 @@ maximum queued service; reverse feedback is unshaped. They are not a model of
public NAT, geographic RTT or every congestion/loss pattern. Reduced quality public NAT, geographic RTT or every congestion/loss pattern. Reduced quality
and temporary stalls under real constraints remain possible. and temporary stalls under real constraints remain possible.
### Balanced Startup And Recovery
Serial Chrome 152 checks on 2026-09-25 reproduced two composition defects in
VP8, 1080p30, motion + balanced, with a 5 Mbps ceiling. A forward-only UDP
shaper limited A to 400 kbps; B remained unshaped. These synthetic-source runs
establish delivered dimensions and recovery, not game perceptual quality or the
cause of reports without diagnostics.
- A group's local warmup spent its five-frame startup protection before any
child published its frames. Restoring balanced then allowed native initial
downscaling against an untrained allocation. Protection now begins at the
first committed outgoing frame; paused/empty frames cannot spend it and later
children cannot reset it. The frame threshold and polling cadence are unchanged.
- The synthetic carrier used motion content intent. Chromium classifies that
as realtime video, without the default screen-content ALR probing. After a
producer adapted downward, low real output could leave the outgoing native
bandwidth estimate slow to recover. The carrier now uses detail once at
construction, selecting the framework's screen-content behavior. The real
producer retains motion and the Host's degradation preference.
Pinned WebRTC explains both boundaries: starting the quality scaler
[restarts initial frame dropping](https://webrtc.googlesource.com/src/+/6f37672d358475cd17544121a12494da454d85fb/video/adaptation/video_stream_encoder_resource_manager.cc#230);
[content hints select the screencast option](https://webrtc.googlesource.com/src/+/6f37672d358475cd17544121a12494da454d85fb/pc/rtp_sender.cc#1423),
which selects [ALR probing configuration](https://webrtc.googlesource.com/src/+/6f37672d358475cd17544121a12494da454d85fb/video/video_send_stream_impl.cc#183).
The [screen probing default is enabled](https://webrtc.googlesource.com/src/+/6f37672d358475cd17544121a12494da454d85fb/rtc_base/experiments/alr_experiment.cc#51).
Detail also changes the tiny encoder's content type; it is not an independent
public probing switch or a request to change the real picture's quality.
| VP8, 14-second constraint / 40-second recovery | Initial A / B | Constrained A / B | Recovered A / B |
| --- | --- | --- | --- |
| Ordinary | 1080p / 1080p | 540p / 1080p | 1080p / 1080p |
| Previous pool | 270p / 270p | 270p / 540p | 720p / 1080p |
| Publication-start correction alone | 1080p / 1080p | 270p / 1080p | 540p / 1080p |
| Publication start + screen-content carrier | 1080p / 1080p | 270p / 1080p | 1080p / 1080p |
The combined VP8 run returned to 1080p about 17 seconds after shaping ended.
H264 with played audio also returned to 1080p; its unconstrained child retained
1080p throughout. A separate one-second pulse after 45 healthy seconds still
caused native downscaling: the repaired VP8/audio path briefly reached 180p,
then regained 1080p about 13 seconds after pulse start. The previous pool's
no-audio pulse ended the 40-second recovery at 360p. Audio changes the bandwidth
composition, so these pulse runs do not establish an exact speedup. Normal
WebRTC can also temporarily reduce resolution after a pulse; this repair does
not promise blur-free delivery under changing network or device load.
A single-consumer control, retaining its mature producer, recovered without
the deeper dual-consumer drop. Together with the publication-start control,
this distinguishes cold producer adaptation from repeated producer churn.
The rate-owner repair separately retains existing encoders through output-rate
spikes and shared budget changes; a genuinely weaker child can still need its
own producer. No delayed-budget policy, bitrate floor or manual recovery probe
was added. Probe padding remains framework-owned traffic under native limits.
Budget attribution remains unchanged: targetBitrate is the encoder's allocated
target, not raw link bandwidth ([upstream stats correction](https://webrtc.googlesource.com/src/+/fe25b0e928ea4e64aa134f5dc8012343320deec5%5E%21/)).
Replacing it with availableOutgoingBitrate would bypass native allocation and
protection. Keep producer, carrier, egress and decoded observations separate.
## Cost And Accounting ## Cost And Accounting
Before the CPU-carrier refinement, a matched actual-product VP8 1080p30 comparison's ordinary pair made Before the CPU-carrier refinement, a matched actual-product VP8 1080p30 comparison's ordinary pair made
@@ -133,6 +191,8 @@ npx tsx scripts/browser-local-pool-probe.ts carrier --1080 --background
``` ```
Add `--h264`, `--single` or `--late` for the corresponding case. Set Add `--h264`, `--single` or `--late` for the corresponding case. Set
`--network --auto --short-pulse` for a 45-second warmup, one-second constraint
and 40-second recovery instead of the ordinary 14-second constraint. Set
`CHROME_PATH` for another installed Chromium binary. `--auto` extends the `CHROME_PATH` for another installed Chromium binary. `--auto` extends the
recovery observation; product code owns all adaptation. Results go to ignored recovery observation; product code owns all adaptation. Results go to ignored
`build/browser-local-pool`; summarize with `build/browser-local-pool`; summarize with
+287
View File
@@ -0,0 +1,287 @@
# Room Voice: Design Assessment
Reviewed 2026-09-24 against public `v1.6.3` (`f423df0a`). This is a proposal,
not an accepted product contract or implementation authorization. The separate
room-interaction experiment supplies candidate lifecycle behavior; it has not
been integrated. [TODO](../todo.md#now) owns the outstanding work and decisions.
## Recommendation
Keep existing room authority and the Host-rooted screen route. Model room voice
as participant-owned microphone publications and independent listening. Chat,
screen playback and voice share admission, not a media lifetime. Do not turn the
screen tree into a conference graph or introduce another room/session store.
There is no equally small solution that simultaneously promises all 21 members
can talk, preserves every P2P-only deployment, and supplies conference-grade
connectivity. Choose the first voice scope before implementing its transport.
For the next technical experiment, assess a small direct group and audio
recapture isolation first. Four total voice participants is a useful test size,
**not an accepted product limit or a demonstrated performance guarantee**.
If full-room voice is required initially, assess a mature conference SFU before
building a mesh intended to grow into one.
## Current Owners And Actual Gaps
| Current owner | Verified behavior | Consequence for voice |
| --- | --- | --- |
| Room store and authenticated signaling | Room admission, participant identity, grant revocation and session replacement | Extend this authority; no voice login or second membership store |
| `HostAudio` in `src/client/media/host-audio.ts` | Source and microphone mixed into one share-scoped track | This is commentary; it cannot supply independent conversation |
| `internal/app/nativehost/audio_mix.go` | Native source/mic PCM share one mixer and output clock | Keep source audio here; avoid duplicating the Host mic across outputs |
| `src/client/media/viewer-audio.ts` | Audio follows the picture's play/pause/mute state | Room speech needs its own playback owner |
| Screen controller and media edges | One Host source, one upstream per Viewer, bounded screen-copy capacity | Voice must not borrow screen generations, first-video-frame proof or copy slots |
| Embedded SFU admission/media | One committed room publication, video-required metadata, share-scoped resource identity | Multiple audio-only publishers need a new capability, not another flag |
The SFU constraints are explicit in `validPublicationMedia` in
`internal/server/sfu/media.go`, `ResourceFence` and `CommitPublication` in
`internal/server/sfu/admission.go`, and the Host/Viewer dispatch in
`internal/server/signal/router_sfu_signal.go`.
[ADR-0013](../adr/0013-embedded-node-local-media.md) embeds Pion/LiveKit components;
it does not embed LiveKit's complete room service.
The interaction candidate separates a retained Host room session from screen
publication. That is a useful foundation, not permission to allocate rooms on
page visit. Preserve the accepted entry: first share creates the room and an
existing room can resume. Retaining opted-in interaction after stopping media is
the candidate lifecycle, not published behavior. A separate voice-only
room-creation flow would be another product decision.
## Transport Choice
| Approach | Benefit | Cost and boundary |
| --- | --- | --- |
| Small direct voice mesh | Keeps media P2P and does not require a reachable central media server | Each sender sends to every listener; needs a total voice-member bound, separate resource accounting and real NAT/device tests |
| Mature multi-publisher SFU | Each speaker uploads once; suitable direction for full-room voice | New publication/subscription integration and central-media availability/privacy decision |
| Host forwarding or mixing hub | Fewer peer relationships | Reintroduces Host dependency, forwarding/mixing and echo ownership; unsuitable for Host-independent conversation |
| Custom voice relay tree or automatic mesh/SFU switching | Potentially combines deployment options | Adds another routing/recovery problem; outside the first scope |
For a full bidirectional mesh, `N` participants need `N*(N-1)/2` pair connections
and up to `N-1` microphone copies per sender: 21 people means 210 pairs and 20
copies; four means six pairs and three copies. These are topology counts, not
bandwidth or CPU measurements. Limiting speakers alone does not bound sender
upload: one speaker and 20 listeners still need 20 copies. The screen's existing
`C=1..3` limit must retain its meaning; any independent voice budget needs explicit
acceptance and combined-load measurement.
Two SFU implementations deserve a bounded comparison: a complete mature room
engine, or a narrow participant-audio adapter around existing media components.
The former adds deployment and potentially competing session ownership; the
latter leaves Piik responsible for admission, subscriptions, recovery and cleanup.
Do not describe the adapter as mature conference behavior merely because its
forwarder uses mature libraries. LiveKit's
[room/participant/track model](https://docs.livekit.io/intro/basics/rooms-participants-tracks/)
and [SFU architecture](https://docs.livekit.io/reference/internals/livekit-sfu/)
show the layers involved.
Select one topology for the first capability. Jitsi's
[two-party P2P design](https://jitsi.org/blog/p2p4121/) retains its bridge connection
and changes transport when group size changes: useful evidence that a hybrid
mode has real lifecycle cost. It is not a reason to copy that complexity here.
Deployment is part of this decision. Demo and App Local/public-link modes are
P2P-only today. Public-link tunneling carries HTTP/WebSocket control traffic,
not a public SFU media endpoint. Existing STUN is not TURN, and existing embedded
UDP media does not provide media TCP/TLS. A full conference engine's
[connectivity options](https://docs.livekit.io/transport/self-hosting/ports-firewall/)
are not inherited by importing its library. This proposal enables no new service
or port and does not change Demo configuration.
## Proposed Responsibility Boundaries
```text
Existing room authority and authenticated signaling
+-- screen publication -> existing P2P/SFU screen route -> picture/source sound
+-- interaction events -> bounded chat/reactions
+-- voice admission -> participant microphone -> selected voice transport
-> independent voice playback
```
These are responsibility boundaries, not a request for a generic media framework
or one new service per line. Keep the first implementation cohesive:
| Responsibility | Owns | Does not own |
| --- | --- | --- |
| Room authority | Who belongs, who may publish/listen, bounds and retirement | OS microphone permission or local volume |
| Participant voice session | Explicit join intent, current admitted publication/subscriptions and their transport lifetime | Screen routes, share generation or a second reconnect policy |
| Microphone input | One acquisition, device choice, local mute, replacement and disposal | Remote permission to turn capture on |
| Voice playback | Received tracks, local volume/person mute, output capability and autoplay recovery | The video's pause/mute state or remote speaking permission |
Do not attach room speech to a screen peer merely to reduce connection count:
screen parent replacement and share retirement have different lifetimes. The
selected voice transport owns its connections and counts them explicitly.
Prefer a Browser-owned room microphone for both roles, including a Host using
App screen capture. This reuses platform speech processing and avoids a second
native voice service. Reuse existing device-selection semantics, not native
device identifiers. Browser microphone acquisition requires a secure context;
plain HTTP LAN pages and unsupported browsers must have truthful capability
presentation. App detection alone does not establish microphone availability.
See the [capture specification](https://www.w3.org/TR/mediacapture-streams/).
Browser voice also does not inherit the native Pion path's port mapping, NAT
assistance or native media fallback just because an App is attached. Measure
voice reachability independently; a working screen path does not prove it.
Room voice and source sound need different audio treatment. Voice uses the
platform's speech/AEC facilities; music/game sound keeps its fidelity settings.
Do not force both through the existing source-audio Opus profile or write a new
voice codec, AEC, PCM subtraction routine or virtual-audio driver.
## Consent, Lifecycle And Presentation
Proposed first permission model: admitted members may explicitly join voice and
then unmute themselves; existing room removal remains the authority boundary.
Start with a closed microphone. Joining may unlock playback, but only explicit
unmute requests input permission. If Host-only speaking approval is required,
specify it before adding a moderation UI. No remote action may start a microphone;
LiveKit also disables [remote unmute by default](https://docs.livekit.io/intro/basics/rooms-participants-tracks/participants/).
| Event | Proposed behavior |
| --- | --- |
| Share start/stop/pause, source or quality change | Voice stays independent |
| Local mic mute | Silence that publication without leaving voice; retain explicit device intent |
| Local person mute or voice volume | Change local listening only; picture volume remains separate |
| Leave voice | Release mic and voice media, remain in room/chat |
| Host leaves page while room service remains | Retire Host voice; other admitted participants may continue |
| App exits in Local/public-link mode | Room service disappears; independent voice cannot outlive its authority |
| Site-mode App media fails | Browser voice may continue while the Site room authority remains |
| Revocation, replacement or room close | Retire affected resources and reject stale completions/signaling |
| Mic permission/device failure | Preserve picture and other voices; show the actual input failure |
| Hidden page or phone interruption | Visibility alone is not leave/mute; report actual suspension/device loss |
Transient signaling loss is not automatically a media failure or a grant
revocation. Current `presence.go` and `route.Controller.DisconnectSession`
distinguish disconnected sessions from confirmed departure; Viewers have bounded
membership grace. Voice must consume that authority owner's transitions, with no
parallel grace timer or indefinite stale-session publication. Before implementation,
define and test the Host case, reservation retention, reconnect re-admission and
revocation during an outage. Do not extrapolate Viewer grace to a Host or reuse
an old session ID as current authority. This is a design gate, not a claim that
screen recovery already implements voice recovery.
One input owner prevents duplicate captures. Retire late permission results after
leave; prepare a replacement device before releasing a healthy old input; keep a
failed explicit selection visible rather than silently choosing another mic.
Repeated mute/join/leave must converge without a second saved effective-state flag.
Ordinary publication mute need not renegotiate every time; mature systems
[distinguish mute, unpublish and unsubscribe](https://docs.livekit.io/intro/basics/rooms-participants-tracks/tracks/).
Use the existing participants/sofa and room actions, not a conference grid.
First controls: join/leave voice, mic toggle and input choice/check, independent
voice volume, and local person mute in the participant menu. Keep source-sound
settings with the picture. Default to system output; show output selection only
where supported. Do not add deafen, global push-to-talk or a permission-role matrix
until their need and restoration behavior are established.
Display local intent, authorized membership, transport readiness and actual
playback separately. A speaking pulse follows observed audio activity, not a
connected flag or UUID animation seed. Catch blocked playback, including tracks
arriving after joining, and offer an explicit enable-audio action; compare
[LiveKit's autoplay handling](https://docs.livekit.io/reference/components/react/component/startaudio/).
Mobile lock-screen calling remains unproven; platform suspension is not fixed by
extra timers. See [browser lifecycle guidance](https://developer.chrome.com/docs/web-platform/page-lifecycle-api).
## Audio Isolation Is A Prerequisite
Whole-system loopback records received room speech played on the captured output
endpoint. That speech can then be sent back through screen audio. Headphones
remove an acoustic path, not this digital recapture; microphone AEC does not clean
the separate source-audio track. This matters even if only the Host shares a screen.
Windows exposes narrower process-tree capture and process exclusion. Current
Piik includes the selected process tree for window sound or captures the output
endpoint for system sound; it does not implement process exclusion. Excluding
the browser that plays Piik voice could also exclude intended browser content.
Test actual process ownership and output routing rather than assuming a window
selection isolates sound. See Microsoft's
[loopback description](https://learn.microsoft.com/en-us/windows/win32/coreaudio/loopback-recording)
and [process-capture sample](https://learn.microsoft.com/en-us/samples/microsoft/windows-classic-samples/applicationloopbackaudio-sample/).
Browser `restrictOwnAudio` is relevant; `suppressLocalAudioPlayback` has a different
purpose. Chrome documents the former as excluding audio originating from the
capturing document when system audio is captured. Check support, applied settings
and decoded output on supported platforms; a hint is not cross-browser proof.
See the [Screen Capture specification](https://www.w3.org/TR/screen-capture/)
and [Chrome's implementation announcement](https://developer.chrome.com/blog/chrome-141-beta).
Verify source sound and incoming voices using distinguishable signals. Cover
native window/system capture, browser tab/window/system capture, source changes,
same-browser content and output-device changes. Where isolation cannot be
established, a supported combination or explicit feature limitation must be
chosen before release. Never silently discard source sound or promise that a
headphone hint solves it.
## Compatibility, Trust And Bounds
The Host microphone must have one distribution path. Sending it in both existing
commentary and independent voice causes duplicates and defeats individual mute.
Moving it exclusively to voice can make older/non-participating Viewers lose
commentary. Choose an explicit room capability/participation boundary or a
coordinated major-version transition; do not ship a silent behavior change.
The [versioning owner](../standards/versioning.md) governs that choice. This
assessment does not select a release number or authorize removal of native mic.
Once that boundary is selected, define the handoff before coding: retire the old
microphone distribution before enabling the new one, and test failure/cancellation
without silently losing commentary for Viewers who cannot receive room voice.
Browser-first capture can avoid new native voice commands, but does not make
room signaling automatically compatible. Trace strict readers in both directions;
advertise support before new messages. Keep stable peer identity, authenticated
session, grant and media connection identity distinct. Reuse an existing identity
only when its lifetime matches; do not add a room-wide epoch for convenience.
Bind signaling to current authenticated participants and exact publication/peer
resources; bound memberships, publications, subscriptions, candidates and message
rates before allocation. Leave/revoke must release actual resources before their
slots become reusable. No audio through chat events or durable voice recordings.
P2P revocation has an enforcement limit: the server can refuse signaling and
conforming endpoints can stop capture/drop tracks, but it cannot intercept media
between two modified peers. An SFU can enforce its own forwarding removal. Do not
promise identical server-enforced muting across these models, or hide the central
server's media access under the existing P2P privacy label.
## Implementation Sequence And Acceptance
1. Reconcile the isolated interaction candidate with current main and accept its
room/picture separation on its own. Preserve the existing staged work; do not
merge an old branch wholesale. Voice need not block chat/UI acceptance.
2. Select total voice membership, deployment coverage, speaking policy and the
commentary/compatibility boundary. Record accepted changes in the existing
product owners and an ADR only when architectural decisions are accepted.
3. Run an isolated transport/audio-isolation experiment before shipping UI.
Measure screen plus voice CPU, upload, number of connections, audible latency,
NAT failure, echo and cleanup. Use one topology; stop if the required scope
implies a custom conference engine or unsupported audio isolation.
4. Implement one vertical slice through the owners above: explicit join, muted
input, unmute, remote audio, local mute and leave. Then expand participants and
exact authority teardown. Reuse existing controls, status/comic meanings and
diagnostic export; avoid separate feature-local rules.
5. Accept faults as well as the happy path: permission resolves after leave,
device switch fails, source changes, autoplay is blocked, signaling drops
during healthy audio, reconnect races revoke/replacement, Host leaves, service
exits, mobile suspends, and repeated join/leave leaks no resources. Test quiet
and muted participants without video-frame admission requirements. Include
supported old/new endpoints and Chinese/English/visual presentation.
Transport experiments must report measured limits rather than treating a
successful localhost call as public-network or mobile acceptance. This planning
round ran no voice, acoustic-echo or device-performance experiment.
## Independent Fidelity Work
Stereo acceptance can proceed separately: distinguishable left/right signals
through Browser, Native, relay and SFU, with microphone mixing and source changes.
The released native answer fix negotiates receive-side stereo; SDP alone does not
prove decoded channel separation. Opus negotiation follows
[RFC 7587 section 7](https://www.rfc-editor.org/rfc/rfc7587.html#section-7).
Do not add surround channels as a prerequisite for voice.
For [HDR issue #420](https://github.com/TNTcraftHIM/Piik/issues/420), investigate
correct SDR output first: preserve HDR range during native capture, then use
platform tone/gamut mapping before the existing SDR encoder. Windows documents
[floating-point capture](https://learn.microsoft.com/en-us/windows/apps/develop/media-authoring-processing/screen-capture)
and a [tone-map effect](https://learn.microsoft.com/en-us/windows/win32/direct2d/hdr-tone-map-effect).
Validate mixed SDR/HDR content, highlights, SDR reference output and display-mode
changes. Already-clipped browser input cannot be restored downstream. Full HDR
transport/metadata, surround sound, recording and transcription remain outside
this proposal.
+1 -1
View File
@@ -24,7 +24,7 @@ service secret store or an untracked access-restricted environment file.
| `LISTEN_HOST` | Defaults to `0.0.0.0`; bare-metal production normally uses `127.0.0.1`. | | `LISTEN_HOST` | Defaults to `0.0.0.0`; bare-metal production normally uses `127.0.0.1`. |
| `PORT` | Positive TCP port, default `8787`; the tracked release wrapper supports only that default. | | `PORT` | Positive TCP port, default `8787`; the tracked release wrapper supports only that default. |
| `PUBLIC_BASE_URL` | Exact public HTTP(S) origin; production requires HTTPS. | | `PUBLIC_BASE_URL` | Exact public HTTP(S) origin; production requires HTTPS. |
| `ALLOWED_ORIGINS` | Comma-separated exact HTTP(S) origins; wildcard is invalid. | | `ALLOWED_ORIGINS` | Unset or empty follows `PUBLIC_BASE_URL`. A non-empty comma-separated list replaces that default; include every trusted browser origin (scheme, host and port). Wildcard is invalid. |
| `SITE_ACCESS_PASSWORD` | Optional in every environment. Unset or empty allows entry without a site password. A configured value is matched exactly, including spaces and Unicode; there are no password length or character rules. General HTTP request limits still apply. Room ownership and Viewer admission remain independent. | | `SITE_ACCESS_PASSWORD` | Optional in every environment. Unset or empty allows entry without a site password. A configured value is matched exactly, including spaces and Unicode; there are no password length or character rules. General HTTP request limits still apply. Room ownership and Viewer admission remain independent. |
| `ROOM_DATABASE_PATH` | Hosted defaults to `rooms.sqlite` in its working directory when unset or blank. An explicit absolute file path selects another SQLite file; `:memory:` opts into process-memory room authority. App Local remains in memory. | | `ROOM_DATABASE_PATH` | Hosted defaults to `rooms.sqlite` in its working directory when unset or blank. An explicit absolute file path selects another SQLite file; `:memory:` opts into process-memory room authority. App Local remains in memory. |
| `MAX_VIEWERS_PER_ROOM` | `1..20`, default `20`; excludes the Host. | | `MAX_VIEWERS_PER_ROOM` | `1..20`, default `20`; excludes the Host. |
+9
View File
@@ -245,6 +245,15 @@ hits from tests and previews, are leads rather than proof of product use or
absence. Follow the actual producer-to-consumer path and check the changed absence. Follow the actual producer-to-consumer path and check the changed
boundary; isolated helper tests can miss a mismatch between the two ends. boundary; isolated helper tests can miss a mismatch between the two ends.
For a shared-resource failure, review both the failing child and a healthy
sibling through the real producer/consumer boundary. Compare preparation with
live operation, and current implementations with supported compatibility paths.
Exercise optional failure after success and a dependency that remains pending
after cancellation; a timeout constant or a close call alone proves no deadline.
Keep the regression at the owning boundary and verify it rejects the old
behavior. Report which failure sequences were exercised, rather than treating
test counts, file coverage or an earlier audit as proof of lifecycle coverage.
Trace acquisition, use, commit and retirement, including replacement and failure. Trace acquisition, use, commit and retirement, including replacement and failure.
For async ownership, check A starting, B replacing it, then A completing or For async ownership, check A starting, B replacing it, then A completing or
cleaning up. Where compatibility is promised or being established for public cleaning up. Where compatibility is promised or being established for public
+19 -6
View File
@@ -64,7 +64,9 @@ production behavior and remaining acceptance.
the selected source produces no frames. Setup and encoding failures remain the selected source produces no frames. Setup and encoding failures remain
bounded; quiet-source waiting must not block pause, stop or connection control. bounded; quiet-source waiting must not block pause, stop or connection control.
Applied settings change only after the replacement is ready and installed; Applied settings change only after the replacement is ready and installed;
failure preserves the previous capture and profile. Platform quality preference uses the hardware required-output failure preserves the previous capture and profile. Failed lower
outputs stay unavailable across preparation and installation without rejecting
a healthy original output. Platform quality preference uses the hardware
encoder's standard quality-versus-speed hint. Pion owns Native WebRTC encoder's standard quality-versus-speed hint. Pion owns Native WebRTC
transport; the shared media adapter uses LiveKit media components for bandwidth transport; the shared media adapter uses LiveKit media components for bandwidth
estimation, forwarding allocation, pacing and recovery. estimation, forwarding allocation, pacing and recovery.
@@ -113,7 +115,9 @@ room quality setting.
Auto uses one bounded actual-sender probe rather than a capability or device Auto uses one bounded actual-sender probe rather than a capability or device
allowlist. A proved sender prefers H.264 with VP8 fallback; unsupported, failed, allowlist. A proved sender prefers H.264 with VP8 fallback; unsupported, failed,
or inconclusive probes use VP8. Manual VP8 or H264 strictly selects that codec. or inconclusive probes use VP8. Manual VP8 or H264 strictly selects that codec.
The result stays fixed for the current share and is reconsidered for a new share. The probe deadline covers setup, negotiation and statistics; cancellation retires
its resources even when a Browser operation remains pending. The result stays
fixed for the current share and is reconsidered for a new share.
ADR-0007 owns probe and negotiation mechanics. ADR-0007 owns probe and negotiation mechanics.
The Browser chooses the concrete encoder implementation. A reported H.264 codec The Browser chooses the concrete encoder implementation. A reported H.264 codec
@@ -130,8 +134,10 @@ choice for quality/source replacement. Native relay encoding uses the same
selection owner. Auto measures encoding work selection owner. Auto measures encoding work
for synthetic NV12 frames at the selected dimensions and frame rate, with a for synthetic NV12 frames at the selected dimensions and frame rate, with a
bounded warmup and sample. If H264 sustains the target it is selected; otherwise bounded warmup and sample. If H264 sustains the target it is selected; otherwise
VP8 is measured within the remaining four-second budget. This is a throughput VP8 is measured within the remaining four-second budget. An unsuccessful or
check, not a perceptual-quality score or a promise under future GPU load. timed-out comparison retains an already-proved H264 encoder; without a proved
encoder, startup still fails. This is a throughput check, not a perceptual-quality
score or a promise under future GPU load.
The returned actual codec owns the shared source, preview, and relay; live The returned actual codec owns the shared source, preview, and relay; live
quality/source changes retain it. Other native platform encoders remain H264. quality/source changes retain it. Other native platform encoders remain H264.
@@ -143,6 +149,10 @@ it is never attached directly to an outbound sender. Replacing or retiring a
sender also stops its clone, so native adaptation state cannot survive by being sender also stops its clone, so native adaptation state cannot survive by being
inherited through the original track. Host pause and live capture constraints inherited through the original track. Host pause and live capture constraints
are propagated to current Host-owned clones. are propagated to current Host-owned clones.
Connection retirement also settles its pending sender and negotiation waits;
closing a Browser PeerConnection alone does not guarantee that its promises
settle. Cancellation starts before joining work that depends on it, while a
replacement still waits for the retired owner's cleanup.
Eligible Browser P2P parents use the source-owned pool in Eligible Browser P2P parents use the source-owned pool in
[ADR-0014](../adr/0014-browser-node-local-encoding-pool.md). Compatible direct [ADR-0014](../adr/0014-browser-node-local-encoding-pool.md). Compatible direct
@@ -151,6 +161,9 @@ remain separate. Each outgoing connection keeps native transport, allocation
and recovery, with a tiny carrier supplying its RTP clock. Producers use and recovery, with a tiny carrier supplying its RTP clock. Producers use
the existing native video target under Host ceilings. Actual forwarded-frame the existing native video target under Host ceilings. Actual forwarded-frame
and producer observations remain distinct from the tiny carrier's statistics. and producer observations remain distinct from the tiny carrier's statistics.
Producer startup protection begins at actual outgoing publication, excluding
local warmup and paused frames. The synthetic carrier uses screen-content
transport probing; the real producer keeps the Host's picture adaptation.
The first eligible consumer follows the same path; unsupported APIs or failed The first eligible consumer follows the same path; unsupported APIs or failed
pooling use ordinary senders. Prepared ordinary quality candidates, Native pooling use ordinary senders. Prepared ordinary quality candidates, Native
ingress and Browser SFU remain independent compositions. ingress and Browser SFU remain independent compositions.
@@ -186,8 +199,8 @@ starts one shared decode/scale/encode group, while healthy forwarding retains
the original input without starting a decoder. The group produces the highest the original input without starting a decoder. The group produces the highest
needed output and its lower fallbacks and stops unused upper outputs. Each child needed output and its lower fallbacks and stops unused upper outputs. Each child
receives only its selected output. The forwarding SFU adds no server transcoder. receives only its selected output. The forwarding SFU adds no server transcoder.
Downstream attachment failure or retirement ends only that edge, never its Downstream attachment or sending failure and retirement belong to that edge,
shared source or healthy siblings. never its shared video/audio source or healthy siblings.
Native capture uses the same screen-share output construction within Host Native capture uses the same screen-share output construction within Host
ceilings. Its source, source clock and higher outputs remain independent of ceilings. Its source, source clock and higher outputs remain independent of
+27 -8
View File
@@ -1,6 +1,6 @@
# Current TODO Ledger # Current TODO Ledger
Last reviewed: 2026-09-24 Last reviewed: 2026-09-25
Only **Now** is executable. Product modules own behavior; Git/PRs own completed Only **Now** is executable. Product modules own behavior; Git/PRs own completed
history. A parked idea is not implementation authority. history. A parked idea is not implementation authority.
@@ -28,20 +28,30 @@ history. A parked idea is not implementation authority.
can resume, and stopping media keeps the opted-in interaction session. Do not can resume, and stopping media keeps the opted-in interaction session. Do not
add visit-triggered creation or a separate pre-share room-creation entry. add visit-triggered creation or a separate pre-share room-creation entry.
This experiment does not authorize integration or publication. This experiment does not authorize integration or publication.
- [ ] **Assess room voice and media fidelity alongside interaction.** Compare - [ ] **Room voice: design before implementation.** The
mature room-audio designs before selecting a small first scope: explicit mic [voice assessment](./research/room-voice.md) maps current owners, mature options
consent, mute/device/output controls, speaker authority and disconnect cleanup. and a staged verification plan. Resolve total voice membership/topology,
Keep room membership, voice publication and the Host's shared picture distinct; P2P-only deployment coverage, speaking authority and the Host-commentary
do not send audio through chat signaling or reverse the current screen tree. compatibility boundary before implementation. Validate source-audio isolation
from received speech before exposing simultaneous sharing and voice. Keep
room authority, microphone and screen lifetimes distinct; no new room store,
screen-tree reversal or automatic mesh/SFU controller is approved.
- [ ] **Verify media fidelity independently of voice.**
Verify actual left/right separation through Browser, Native and SFU paths; Verify actual left/right separation through Browser, Native and SFU paths;
receive-side stereo negotiation alone does not establish audible fidelity. receive-side stereo negotiation alone does not establish audible fidelity.
Assess HDR capture to correct SDR output for ordinary viewers before full HDR, Assess HDR capture to correct SDR output for ordinary viewers before full HDR,
with issue [#420](https://github.com/TNTcraftHIM/Piik/issues/420) as the reported with issue [#420](https://github.com/TNTcraftHIM/Piik/issues/420) as the reported
case. Multi-speaker routing, full HDR and surround sound remain design decisions, case. Full HDR and surround sound remain design decisions. These findings do
not additions authorized for this maintenance release. not establish physical device acceptance or authorize a new release.
## Awaiting Device Or Reporter Evidence ## Awaiting Device Or Reporter Evidence
- [ ] **Self-hosted room creation HTTP 403.** Retest affected deployments using
the [address checks](./guide/troubleshooting.md#room-creation-returns-403).
A stale explicit `ALLOWED_ORIGINS` overrides `PUBLIC_BASE_URL`; the sample now
follows the public address and Piik identifies its Origin rejection. Reporter
configuration and response details are still absent; distinguish Piik's
rejection from a proxy/WAF 403 before assigning their cause.
- [ ] **No available media route.** A decline is reported since around v1.4; - [ ] **No available media route.** A decline is reported since around v1.4;
establish comparable attempts on the same endpoints/network before attributing establish comparable attempts on the same endpoints/network before attributing
a version regression. Obtain matched Host/Viewer diagnostics from a version regression. Obtain matched Host/Viewer diagnostics from
@@ -97,6 +107,9 @@ history. A parked idea is not implementation authority.
local permission and recovery checks do not establish the reporters' causes. local permission and recovery checks do not establish the reporters' causes.
Manual H264 returning immediately to idle also needs reporter diagnostics; Manual H264 returning immediately to idle also needs reporter diagnostics;
bounded selection past an unusable hardware encoder is locally verified. bounded selection past an unusable hardware encoder is locally verified.
Startup failure is reported again on the latest release without a known codec.
Auto selection now retains proved H264 if the optional VP8 comparison times
out; this confirmed selection defect does not establish the reporter's cause.
A separate v1.6.1 App/Chrome report lists Window/Screen sources, then waits A separate v1.6.1 App/Chrome report lists Window/Screen sources, then waits
and returns to idle with a share-start error after selection. Its codec is and returns to idle with a share-start error after selection. Its codec is
unknown; similarity to the H264 symptom does not establish the same cause. unknown; similarity to the H264 symptom does not establish the same cause.
@@ -110,6 +123,12 @@ history. A parked idea is not implementation authority.
fullscreen, display-mode change and device loss. A reproduced downstream fullscreen, display-mode change and device loss. A reproduced downstream
retirement race no longer stops the shared source. Controlled-window checks retirement race no longer stops the shared source. Controlled-window checks
do not reproduce the reporting game/device environment. do not reproduce the reporting game/device environment.
- [ ] **Brief blur followed by immediate recovery.** Compare the same source,
codec, peers and network with ordinary WebRTC before attributing the report
to pooling. The [pool checks](./research/browser-local-encoding-pool.md#balanced-startup-and-recovery)
cover rate-owner retention, publication startup, native recovery probing and
ordinary fallback. They do not establish this reporter's capture path or result;
correlate actual receiver dimensions, QP, resource adaptation and packet loss.
- [ ] **Windows 32-bit candidate acceptance.** Verify the isolated - [ ] **Windows 32-bit candidate acceptance.** Verify the isolated
`spike/windows-x86-capture` candidate's launch, capture/audio, memory pressure, `spike/windows-x86-capture` candidate's launch, capture/audio, memory pressure,
source replacement and update links on a 32-bit Windows device. WOW64 source replacement and update links on a 32-bit Windows device. WOW64
+4 -1
View File
@@ -114,7 +114,10 @@ func (source *AudioSource) WritePCMWithDropped(pcm []byte, duration time.Duratio
return err return err
} }
source.bytes.Add(uint64(len(packet))) source.bytes.Add(uint64(len(packet)))
return source.samples.WriteSample(media.Sample{Data: packet, Duration: duration, PrevDroppedPackets: dropped}) // Pion still delivers to healthy bindings when one connection cannot write.
// Connection lifecycle owns those failures; they do not end the PCM source.
_ = source.samples.WriteSample(media.Sample{Data: packet, Duration: duration, PrevDroppedPackets: dropped})
return nil
} }
func (source *AudioSource) WriteRTP(packet *rtp.Packet) error { func (source *AudioSource) WriteRTP(packet *rtp.Packet) error {
+75
View File
@@ -0,0 +1,75 @@
package mediaedge
import (
"io"
"testing"
"time"
"github.com/TNTcraftHIM/Piik/internal/app/nativeaudio"
"github.com/pion/interceptor"
"github.com/pion/rtp"
"github.com/pion/webrtc/v4"
)
// Exercise Pion's actual shared-track fanout without sockets or retirement races.
type audioBinding struct {
id string
err error
packets int
}
func (binding *audioBinding) CodecParameters() []webrtc.RTPCodecParameters {
return []webrtc.RTPCodecParameters{{RTPCodecCapability: opusCapability, PayloadType: 111}}
}
func (*audioBinding) HeaderExtensions() []webrtc.RTPHeaderExtensionParameter { return nil }
func (*audioBinding) SSRC() webrtc.SSRC { return 1 }
func (*audioBinding) SSRCRetransmission() webrtc.SSRC { return 0 }
func (*audioBinding) SSRCForwardErrorCorrection() webrtc.SSRC { return 0 }
func (binding *audioBinding) WriteStream() webrtc.TrackLocalWriter { return binding }
func (binding *audioBinding) ID() string { return binding.id }
func (*audioBinding) RTCPReader() interceptor.RTCPReader { return nil }
func (binding *audioBinding) WriteRTP(_ *rtp.Header, payload []byte) (int, error) {
binding.packets++
return len(payload), binding.err
}
func (binding *audioBinding) Write(payload []byte) (int, error) {
return binding.WriteRTP(nil, payload)
}
func TestAudioSourceIsolatesBindingWriteFailure(t *testing.T) {
engine, err := NewEngine(EngineOptions{BindAddress: "127.0.0.1:0", IncludeLoopback: true})
if err != nil {
t.Fatal(err)
}
defer engine.Close()
source, err := engine.NewAudioSource(2, 128_000)
if err != nil {
t.Fatal(err)
}
defer source.Close()
failed := &audioBinding{id: "retiring", err: io.ErrClosedPipe}
healthy := &audioBinding{id: "healthy"}
for _, binding := range []*audioBinding{failed, healthy} {
if _, err = source.samples.Bind(binding); err != nil {
t.Fatal(err)
}
}
pcm := make([]byte, nativeaudio.FrameBytes)
for expected := 1; expected <= 2; expected++ {
if err = source.WritePCM(pcm, 20*time.Millisecond); err != nil {
t.Fatalf("one failed binding stopped the shared source: %v", err)
}
if healthy.packets != expected || failed.packets != expected {
t.Fatalf("fanout stopped: healthy=%d failed=%d", healthy.packets, failed.packets)
}
}
if err = source.WritePCM(pcm[:1], 20*time.Millisecond); err == nil {
t.Fatal("invalid PCM was accepted")
}
if err = source.Close(); err != nil {
t.Fatal(err)
}
if err = source.WritePCM(pcm, 20*time.Millisecond); err == nil {
t.Fatal("closed audio source was accepted")
}
}
+7 -2
View File
@@ -42,8 +42,13 @@ func (engine *Engine) NewPublication(source *Source, options EdgeOptions) (*Publ
len(options.ConnectionID) > 256 || options.Audio != nil && options.Audio.engine != engine { len(options.ConnectionID) > 256 || options.Audio != nil && options.Audio.engine != engine {
return nil, errors.New("native publication input is invalid") return nil, errors.New("native publication input is invalid")
} }
// A new publication starts on the base source. Independent lower-output
// groups are not part of its descriptor and own their failures separately.
outputsUnavailable := func() bool {
return slices.Contains(source.outputBitrates[:min(len(source.outputBitrates), len(source.formats))], uint32(0))
}
source.mu.Lock() source.mu.Lock()
unavailable := slices.Contains(source.outputBitrates, uint32(0)) unavailable := outputsUnavailable()
source.mu.Unlock() source.mu.Unlock()
if unavailable { if unavailable {
return nil, errors.New("native publication output is unavailable") return nil, errors.New("native publication output is unavailable")
@@ -89,7 +94,7 @@ func (engine *Engine) NewPublication(source *Source, options EdgeOptions) (*Publ
}) })
engine.mu.Lock() engine.mu.Lock()
source.mu.Lock() source.mu.Lock()
if engine.closed || source.closed || len(source.publications) >= 2 || slices.Contains(source.outputBitrates, uint32(0)) { if engine.closed || source.closed || len(source.publications) >= 2 || outputsUnavailable() {
source.mu.Unlock() source.mu.Unlock()
engine.mu.Unlock() engine.mu.Unlock()
_ = publication.Close() _ = publication.Close()
@@ -97,6 +97,47 @@ func TestPublicationOwnsOneReservationAndSourceLifetime(t *testing.T) {
} }
} }
func TestPublicationAdmissionIgnoresUnrelatedFailedOutputs(t *testing.T) {
engine, err := NewEngine(EngineOptions{BindAddress: "127.0.0.1:0", IncludeLoopback: true})
if err != nil {
t.Fatal(err)
}
defer engine.Close()
source, err := engine.NewSource("vp8", 2, 2, nil)
if err != nil {
t.Fatal(err)
}
defer source.Close()
publication, err := engine.NewPublication(source, EdgeOptions{ConnectionID: "initial"})
if err != nil {
t.Fatalf("unconfigured outputs prevented initial publication: %v", err)
}
defer publication.Close()
if err = source.ConfigureOutputs([]uint32{90_000, 300_000, 90_000, 90_000}); err != nil {
t.Fatal(err)
}
for _, slot := range []int{2, 3, 0, 1} {
if err = source.DisableLayer(slot); err != nil {
t.Fatal(err)
}
candidate, createErr := engine.NewPublication(source, EdgeOptions{ConnectionID: "replacement"})
if slot >= 2 {
if createErr != nil {
t.Fatalf("unrelated failed output %d prevented publication: %v", slot, createErr)
}
if len(candidate.Media().Layers) != 2 || len(source.publications) != 2 {
t.Fatal("independent output failure changed the base publication")
}
if err = candidate.Close(); err != nil {
t.Fatal(err)
}
} else if createErr == nil {
_ = candidate.Close()
t.Fatalf("failed base output %d was admitted", slot)
}
}
}
func TestPublicationQualityUsesHighestSentLayerCadence(t *testing.T) { func TestPublicationQualityUsesHighestSentLayerCadence(t *testing.T) {
check := func(err error) { check := func(err error) {
t.Helper() t.Helper()
+37 -10
View File
@@ -39,6 +39,8 @@ type CaptureState struct {
FPS uint32 `json:"fps,omitempty"` FPS uint32 `json:"fps,omitempty"`
RestoreToken string `json:"restoreToken,omitempty"` RestoreToken string `json:"restoreToken,omitempty"`
Outputs []nativecapture.OutputProfile `json:"outputs"` Outputs []nativecapture.OutputProfile `json:"outputs"`
unavailableLayers []int
} }
type Event struct { type Event struct {
@@ -91,14 +93,17 @@ type Session struct {
done chan error done chan error
ready chan error ready chan error
mu sync.Mutex mu sync.Mutex
updateMu sync.Mutex updateMu sync.Mutex
edges map[string]*mediaedge.Edge edges map[string]*mediaedge.Edge
publications map[publicationKey]*mediaedge.Publication publications map[publicationKey]*mediaedge.Publication
captureApplied chan struct{}
audioChanged chan struct{} captureApplied chan struct{}
paused bool captureUnavailable []int
closed bool
audioChanged chan struct{}
paused bool
closed bool
} }
func Start(parent context.Context, options Options) (*Session, error) { func Start(parent context.Context, options Options) (*Session, error) {
@@ -512,6 +517,7 @@ func (session *Session) commitCapture(
session.stream = replacement session.stream = replacement
applied := make(chan struct{}) applied := make(chan struct{})
session.captureApplied = applied session.captureApplied = applied
session.captureUnavailable = state.unavailableLayers
if replaceAudio { if replaceAudio {
session.audioStream = replacementAudio session.audioStream = replacementAudio
select { select {
@@ -545,14 +551,25 @@ func (session *Session) commitCapture(
// Only the video reader installs source metadata, after leaving the old input. // Only the video reader installs source metadata, after leaving the old input.
// The update response waits for this exact stream's installation, not its process. // The update response waits for this exact stream's installation, not its process.
func (session *Session) installCapture(next *nativecapture.Stream) error { func (session *Session) installCapture(next *nativecapture.Stream) error {
session.mu.Lock()
unavailable := session.captureUnavailable
session.mu.Unlock()
session.source.BeginGeneration() session.source.BeginGeneration()
if err := configureCaptureOutputs(session.source, next.Outputs()); err != nil { if err := configureCaptureOutputs(session.source, next.Outputs()); err != nil {
return err return err
} }
// Preparation has consumed these events. Apply them to the new generation
// before acknowledging it, just as the live reader handles unavailable layers.
for _, layer := range unavailable {
if err := session.source.DisableLayer(layer); err != nil {
return err
}
}
session.mu.Lock() session.mu.Lock()
if session.stream == next && session.captureApplied != nil { if session.stream == next && session.captureApplied != nil {
close(session.captureApplied) close(session.captureApplied)
session.captureApplied = nil session.captureApplied = nil
session.captureUnavailable = nil
} }
session.mu.Unlock() session.mu.Unlock()
session.emit(Event{Type: "capture-state", ShareID: session.shareID, State: "active"}) session.emit(Event{Type: "capture-state", ShareID: session.shareID, State: "active"})
@@ -915,6 +932,9 @@ func waitForCaptureProfile(
} }
inputSeen := false inputSeen := false
var starting CaptureState var starting CaptureState
var unavailable []int
outputs := stream.Outputs()
original := min(1, len(outputs)-1)
for { for {
frame, err := stream.Read() frame, err := stream.Read()
if err != nil { if err != nil {
@@ -922,8 +942,14 @@ func waitForCaptureProfile(
return return
} }
if frame.Kind == nativecapture.FrameLayerUnavailable { if frame.Kind == nativecapture.FrameLayerUnavailable {
send(result{stage: "output-unavailable", err: errors.New("native capture profile has an unavailable output")}) if starting.State != "starting" || frame.Layer < 0 || frame.Layer >= len(outputs) || frame.Layer == original {
return send(result{stage: "output-unavailable", err: errors.New("native capture profile has an invalid or unavailable original output")})
return
}
if !slices.Contains(unavailable, frame.Layer) {
unavailable = append(unavailable, frame.Layer)
}
continue
} }
if frame.Kind == nativecapture.FrameBegin && !inputSeen { if frame.Kind == nativecapture.FrameBegin && !inputSeen {
inputSeen = true inputSeen = true
@@ -958,6 +984,7 @@ func waitForCaptureProfile(
return return
} }
state.AdapterIndex, state.EncoderIndex = starting.AdapterIndex, starting.EncoderIndex state.AdapterIndex, state.EncoderIndex = starting.AdapterIndex, starting.EncoderIndex
state.unavailableLayers = unavailable
send(result{state: state}) send(result{state: state})
return return
} }
+68 -14
View File
@@ -8,6 +8,8 @@ import (
"io" "io"
"log/slog" "log/slog"
"os" "os"
"slices"
"strconv"
"strings" "strings"
"testing" "testing"
"time" "time"
@@ -20,6 +22,10 @@ func TestMain(tests *testing.M) {
if os.Getenv("PIIK_NATIVEHOST_PIPE_FIXTURE") == "backend" { if os.Getenv("PIIK_NATIVEHOST_PIPE_FIXTURE") == "backend" {
for _, key := range []string{"PIIK_CAPTURE_STARTING", "PIIK_CAPTURE_ACTIVE"} { for _, key := range []string{"PIIK_CAPTURE_STARTING", "PIIK_CAPTURE_ACTIVE"} {
writeCaptureFixtureFrame(nativecapture.Frame{Kind: nativecapture.FrameStatus, Data: []byte(os.Getenv(key))}) writeCaptureFixtureFrame(nativecapture.Frame{Kind: nativecapture.FrameStatus, Data: []byte(os.Getenv(key))})
if layer := os.Getenv("PIIK_CAPTURE_UNAVAILABLE"); key == "PIIK_CAPTURE_STARTING" && layer != "" {
value, _ := strconv.Atoi(layer)
writeCaptureFixtureFrame(nativecapture.Frame{Kind: nativecapture.FrameLayerUnavailable, Layer: value, Data: []byte("fixture")})
}
} }
_, _ = io.Copy(io.Discard, os.Stdin) _, _ = io.Copy(io.Discard, os.Stdin)
os.Exit(0) os.Exit(0)
@@ -83,7 +89,7 @@ func TestCaptureCommitWaitsForReaderMetadataOrTermination(t *testing.T) {
go func() { go func() {
session.updateMu.Lock() session.updateMu.Lock()
err := session.commitCapture(options, QualityProfile{Video: options.Profile, AudioBitrate: 64_000}, replacement, err := session.commitCapture(options, QualityProfile{Video: options.Profile, AudioBitrate: 64_000}, replacement,
CaptureState{AdapterIndex: &adapter, EncoderIndex: &encoder}, nil, false) CaptureState{AdapterIndex: &adapter, EncoderIndex: &encoder, unavailableLayers: []int{0}}, nil, false)
session.updateMu.Unlock() session.updateMu.Unlock()
committed <- err committed <- err
}() }()
@@ -116,6 +122,35 @@ func TestCaptureCommitWaitsForReaderMetadataOrTermination(t *testing.T) {
if media.Layers[1].Width != 854 || media.Layers[1].Height != 480 || media.Layers[1].Bitrate != 2_000_000 { if media.Layers[1].Width != 854 || media.Layers[1].Height != 480 || media.Layers[1].Bitrate != 2_000_000 {
t.Fatalf("acknowledged stale metadata: %+v", media) t.Fatalf("acknowledged stale metadata: %+v", media)
} }
plan, err := source.BeginFrame(time.Second)
check(err)
if plan.Bitrates[0] != 0 || plan.Bitrates[1] == 0 || session.captureUnavailable != nil {
t.Fatalf("replacement lost or retained pending failures: %+v", plan)
}
// The next healthy generation must not inherit the failed layer.
healthy, err := nativecapture.StartVideo(ctx, executable, options)
check(err)
defer healthy.Close()
go func() {
committed <- session.commitCapture(options, session.profile, healthy, CaptureState{}, nil, false)
}()
select {
case <-replacement.Done():
case <-time.After(5 * time.Second):
t.Fatal("replacement fixture did not exit")
}
check(session.installCapture(healthy))
select {
case err = <-committed:
check(err)
case <-time.After(time.Second):
t.Fatal("healthy replacement remained pending")
}
plan, err = source.BeginFrame(time.Second)
check(err)
if plan.Bitrates[0] == 0 || plan.Bitrates[1] == 0 {
t.Fatalf("healthy generation inherited a failed layer: %+v", plan)
}
return return
} }
go session.run() go session.run()
@@ -165,19 +200,38 @@ func TestCaptureProfileRetainsSelectedBackendFromStarting(t *testing.T) {
t.Fatal(err) t.Fatal(err)
} }
profile := nativecapture.VideoProfile{Width: 1280, Height: 720, Framerate: 30, Bitrate: 3_000_000, Preference: "balanced"} profile := nativecapture.VideoProfile{Width: 1280, Height: 720, Framerate: 30, Bitrate: 3_000_000, Preference: "balanced"}
stream, err := nativecapture.StartVideo(t.Context(), executable, nativecapture.VideoOptions{ for _, failed := range []string{"", "0", "1", "5", "6"} {
Target: nativecapture.CaptureTarget{Kind: "display", SourceID: "1", Title: "Fixture"}, Codec: "h264", Profile: profile, OutputGroups: 4, t.Run("unavailable-"+failed, func(t *testing.T) {
}) t.Setenv("PIIK_CAPTURE_UNAVAILABLE", failed)
if err != nil { stream, err := nativecapture.StartVideo(t.Context(), executable, nativecapture.VideoOptions{
t.Fatal(err) Target: nativecapture.CaptureTarget{Kind: "display", SourceID: "1", Title: "Fixture"}, Codec: "h264", Profile: profile, OutputGroups: 4,
} })
defer stream.Close() if err != nil {
state, err := waitForCaptureProfile(t.Context(), stream, profile, "h264", false, false) t.Fatal(err)
if err != nil { }
t.Fatal(err) defer stream.Close()
} state, err := waitForCaptureProfile(t.Context(), stream, profile, "h264", false, false)
if state.AdapterIndex == nil || *state.AdapterIndex != 1 || state.EncoderIndex == nil || *state.EncoderIndex != 2 { if failed == "1" || failed == "6" {
t.Fatalf("active profile lost the actual hardware selection: %+v", state) if err == nil {
t.Fatal("original or invalid output failure was accepted")
}
return
}
if err != nil {
t.Fatal(err)
}
if state.AdapterIndex == nil || *state.AdapterIndex != 1 || state.EncoderIndex == nil || *state.EncoderIndex != 2 {
t.Fatalf("active profile lost the actual hardware selection: %+v", state)
}
var unavailable []int
if failed != "" {
layer, _ := strconv.Atoi(failed)
unavailable = []int{layer}
}
if !slices.Equal(state.unavailableLayers, unavailable) {
t.Fatalf("preparation lost optional output failures: %+v", state.unavailableLayers)
}
})
} }
} }
+43 -3
View File
@@ -634,7 +634,7 @@ func TestLocalPasswordUsesConfiguredDestinationCookieAcrossLANAndPublicLink(t *t
func TestSiteAccessRequiresAnAllowedOriginAndAValidLoginBody(t *testing.T) { func TestSiteAccessRequiresAnAllowedOriginAndAValidLoginBody(t *testing.T) {
server := start(t, Options{Config: testConfig(t)}) server := start(t, Options{Config: testConfig(t)})
forbidden := `{"error":"Forbidden"}` forbidden := `{"error":"Origin not allowed"}`
server.do(http.MethodPost, "/api/site-access", withBearer(testAccessPassword)). server.do(http.MethodPost, "/api/site-access", withBearer(testAccessPassword)).
expect(http.StatusForbidden, forbidden) expect(http.StatusForbidden, forbidden)
@@ -1085,7 +1085,7 @@ func TestRoomAccessRequiresSameOriginSiteAccessAndTheExactHostToken(t *testing.T
server.updateRoomAccess(accessRequest{ server.updateRoomAccess(accessRequest{
roomID: first.RoomID, hostToken: first.HostToken, body: rotate, roomID: first.RoomID, hostToken: first.HostToken, body: rotate,
cookie: cookie, origin: "https://foreign.test", cookie: cookie, origin: "https://foreign.test",
}).expect(http.StatusForbidden, `{"error":"Forbidden"}`) }).expect(http.StatusForbidden, `{"error":"Origin not allowed"}`)
server.updateRoomAccess(accessRequest{ server.updateRoomAccess(accessRequest{
roomID: first.RoomID, body: rotate, cookie: cookie, roomID: first.RoomID, body: rotate, cookie: cookie,
@@ -1122,7 +1122,7 @@ func TestRoomCreationRejectsMalformedRequestsAndForeignBrowserOrigins(t *testing
server.do(http.MethodPost, "/api/rooms", server.do(http.MethodPost, "/api/rooms",
withOrigin("https://foreign.test"), withJSON(`{"codeEntryPolicy":"open"}`)). withOrigin("https://foreign.test"), withJSON(`{"codeEntryPolicy":"open"}`)).
expect(http.StatusForbidden, `{"error":"Forbidden"}`) expect(http.StatusForbidden, `{"error":"Origin not allowed"}`)
// An oversized or non-JSON body is the same 400. // An oversized or non-JSON body is the same 400.
server.do(http.MethodPost, "/api/rooms", withOrigin(allowedOrigin), server.do(http.MethodPost, "/api/rooms", withOrigin(allowedOrigin),
@@ -1137,6 +1137,46 @@ func TestRoomCreationRejectsMalformedRequestsAndForeignBrowserOrigins(t *testing
expectHeader("Allow", "POST") expectHeader("Allow", "POST")
} }
func TestRoomCreationBehindProxyUsesConfiguredBrowserOrigin(t *testing.T) {
for _, test := range []struct {
name, publicURL, origins, browserOrigin string
status int
}{
{"public default", "https://share.example.test", "", "https://share.example.test", http.StatusCreated},
{"normalized default port", "https://share.example.test:443", "", "https://share.example.test", http.StatusCreated},
{"stale localhost override", "https://share.example.test", "http://localhost:8787", "https://share.example.test", http.StatusForbidden},
{"explicit alternate origin", "https://share.example.test", "https://alternate.test", "https://alternate.test", http.StatusCreated},
{"different public port", "https://share.example.test:8443", "", "https://share.example.test", http.StatusForbidden},
{"missing origin", "https://share.example.test", "", "", http.StatusForbidden},
{"foreign origin", "https://share.example.test", "", "https://foreign.test", http.StatusForbidden},
} {
t.Run(test.name, func(t *testing.T) {
configuration, err := config.Load(map[string]string{
"PUBLIC_BASE_URL": test.publicURL, "ALLOWED_ORIGINS": test.origins,
"ROOM_DATABASE_PATH": ":memory:", "LISTEN_HOST": "127.0.0.1",
})
if err != nil {
t.Fatal(err)
}
configuration.Port = 0
server := start(t, Options{Config: configuration})
proxy := func(request *http.Request) {
request.Host = "127.0.0.1:8787"
request.Header.Set("X-Forwarded-Host", "share.example.test")
request.Header.Set("X-Forwarded-Proto", "https")
}
server.do(http.MethodGet, "/api/site-access", proxy).
expect(http.StatusOK, `{"required":false,"authenticated":true}`)
result := server.do(http.MethodPost, "/api/rooms", proxy,
withOrigin(test.browserOrigin), withJSON(`{"codeEntryPolicy":"open"}`))
result.expectStatus(test.status)
if test.status == http.StatusForbidden {
result.expect(test.status, `{"error":"Origin not allowed"}`)
}
})
}
}
func TestRoomCreationReturnsServiceUnavailableAtTheGlobalRoomBound(t *testing.T) { func TestRoomCreationReturnsServiceUnavailableAtTheGlobalRoomBound(t *testing.T) {
configuration := testConfig(t) configuration := testConfig(t)
configuration.SiteAccessPassword = "" configuration.SiteAccessPassword = ""
+3 -3
View File
@@ -166,7 +166,7 @@ func (s *Server) handleSiteAccess(writer http.ResponseWriter, request *http.Requ
} }
if !s.allowedRequestOrigin(request) { if !s.allowedRequestOrigin(request) {
sendJSON(writer, http.StatusForbidden, errorBody{"Forbidden"}) sendJSON(writer, http.StatusForbidden, errorBody{"Origin not allowed"})
return return
} }
// Released clients send a bodyless Bearer request. JSON preserves passwords // Released clients send a bodyless Bearer request. JSON preserves passwords
@@ -289,7 +289,7 @@ func (s *Server) handleRoomCreation(
return return
} }
if !s.allowedRequestOrigin(request) { if !s.allowedRequestOrigin(request) {
sendJSON(writer, http.StatusForbidden, errorBody{"Forbidden"}) sendJSON(writer, http.StatusForbidden, errorBody{"Origin not allowed"})
return return
} }
// When site access requires a password, room creation uses its cookie; // When site access requires a password, room creation uses its cookie;
@@ -337,7 +337,7 @@ func (s *Server) authorizedRoomHostToken(
request *http.Request, request *http.Request,
) (string, bool) { ) (string, bool) {
if !s.allowedRequestOrigin(request) { if !s.allowedRequestOrigin(request) {
sendJSON(writer, http.StatusForbidden, errorBody{"Forbidden"}) sendJSON(writer, http.StatusForbidden, errorBody{"Origin not allowed"})
return "", false return "", false
} }
if !s.roomCreationAuthorized(request) { if !s.roomCreationAuthorized(request) {
+20 -10
View File
@@ -1265,6 +1265,20 @@ struct VideoEncoderSelection final {
std::unique_ptr<VideoEncoder> initial; std::unique_ptr<VideoEncoder> initial;
}; };
template <typename Measure>
VideoEncoderSelection CompareSoftwareEncoder(std::unique_ptr<VideoEncoder> hardware,
std::optional<double> hardware_work, EncoderClock::time_point deadline, Measure measure) {
try {
RequireEncoderTime(deadline);
const double software_work = measure();
if (!hardware_work || software_work <= *hardware_work) return {};
} catch (const std::exception&) {
if (!hardware_work) throw;
}
// The deadline bounds new probing, not reuse of an already-proved encoder.
return {OutputKind::h264, std::move(hardware)};
}
using EncoderCandidate = std::pair<UINT, UINT>; using EncoderCandidate = std::pair<UINT, UINT>;
void LogEncoderRejection(EncoderCandidate candidate, const GateFailure& error) { void LogEncoderRejection(EncoderCandidate candidate, const GateFailure& error) {
@@ -1353,17 +1367,13 @@ VideoEncoderSelection SelectVideoEncoder(
hardware.reset(); hardware.reset();
device = DeviceContext{}; device = DeviceContext{};
} }
RequireEncoderTime(deadline); if (!device.device) {
if (!device.device) device = CreateDevice(SelectAdapter(adapters, arguments.adapter_index)); RequireEncoderTime(deadline);
try { device = CreateDevice(SelectAdapter(adapters, arguments.adapter_index));
const double software_work = MeasureEncoderWork(nullptr, device.device.Get(),
arguments.profile, deadline);
if (!hardware_work || software_work <= *hardware_work) return {};
} catch (const std::exception&) {
if (!hardware_work) throw;
} }
RequireEncoderTime(deadline); return CompareSoftwareEncoder(std::move(hardware), hardware_work, deadline, [&] {
return {OutputKind::h264, std::move(hardware)}; return MeasureEncoderWork(nullptr, device.device.Get(), arguments.profile, deadline);
});
} }
struct CaptureInput final { struct CaptureInput final {
@@ -149,8 +149,47 @@ void CheckEncoderCandidates() {
assert(tried.empty()); assert(tried.empty());
} }
void CheckAutoEncoderFallback() {
const auto deadline = EncoderClock::time_point::max();
const auto expired = EncoderClock::time_point::min();
auto hardware = [] { return std::make_unique<FixtureEncoder>(nullptr, std::shared_future<void>{}); };
for (const auto probe_deadline : {deadline, expired}) {
auto initial = hardware();
const auto* proven = initial.get();
bool probed = false;
const auto selected = CompareSoftwareEncoder(std::move(initial), .04, probe_deadline, [&]() -> double {
probed = true;
// A slow software comparison exhausts its budget after H264 succeeded.
RequireEncoderTime(expired);
return 0;
});
assert(selected.kind == OutputKind::h264 && selected.initial.get() == proven);
assert(probed == (probe_deadline == deadline));
}
for (const double software_work : {.02, .04, .06}) {
const auto selected = CompareSoftwareEncoder(hardware(), .04, deadline, [&] { return software_work; });
assert((selected.kind == OutputKind::vp8) == (software_work <= .04));
assert(static_cast<bool>(selected.initial) == (software_work > .04));
}
const auto software_only = CompareSoftwareEncoder(nullptr, std::nullopt, deadline, [] { return .03; });
assert(software_only.kind == OutputKind::vp8 && !software_only.initial);
for (const auto probe_deadline : {deadline, expired}) {
bool failed = false;
try {
CompareSoftwareEncoder(nullptr, std::nullopt, probe_deadline, [&]() -> double {
RequireEncoderTime(expired);
return 0;
});
} catch (const GateFailure& error) {
failed = error.stage() == "codec-probe-timeout";
}
assert(failed); // No proved encoder must remain a startup failure.
}
}
int main() { int main() {
CheckEncoderCandidates(); CheckEncoderCandidates();
CheckAutoEncoderFallback();
CheckPrimaryFailureDetails(); CheckPrimaryFailureDetails();
ComPtr<ID3D11Device> device; ComPtr<ID3D11Device> device;
Check(D3D11CreateDevice(nullptr, D3D_DRIVER_TYPE_WARP, nullptr, 0, nullptr, 0, Check(D3D11CreateDevice(nullptr, D3D_DRIVER_TYPE_WARP, nullptr, 0, nullptr, 0,
+6
View File
@@ -208,6 +208,10 @@ async function main(): Promise<void> {
await click(host, 'button[aria-label^="H264"]'); await click(host, 'button[aria-label^="H264"]');
stage = "share-picker"; stage = "share-picker";
await click(host, 'button.lr-tv-big.is-action'); await click(host, 'button.lr-tv-big.is-action');
// This arm measures Native ingress. Browser capture can start without App
// discovery, so first establish the control connection this gate requires.
await waitForSample(() => read(host),
(sample) => sample.controlOpen && sample.requests.includes("list-sources"), 10000);
stage = "share-browser-capture"; stage = "share-browser-capture";
await click(host, 'button[data-source-tab="browser"]'); await click(host, 'button[data-source-tab="browser"]');
await click(host, '.lr-source-option.is-browser'); await click(host, '.lr-source-option.is-browser');
@@ -263,6 +267,8 @@ async function main(): Promise<void> {
stage = "source-audio-change"; stage = "source-audio-change";
await evaluate(cdp, host, `fanoutGate.audio(${!initialAudio})`, Date.now()+5000); await evaluate(cdp, host, `fanoutGate.audio(${!initialAudio})`, Date.now()+5000);
await click(host, 'button[aria-label="Switch source"]'); await click(host, 'button[aria-label="Switch source"]');
await click(host, 'button[data-source-tab="browser"]');
await click(host, '.lr-source-option.is-browser');
await waitForSample(() => read(host), (s) => s.receiveRequests === 2 && s.captures === 2 && s.sharing, 15000); await waitForSample(() => read(host), (s) => s.receiveRequests === 2 && s.captures === 2 && s.sharing, 15000);
const beforeSource = await read(viewers[1]!); const beforeSource = await read(viewers[1]!);
await waitForSample(() => read(viewers[1]!), (s) => s.width > 0 && s.frames >= beforeSource.frames+20 && s.audio===Number(!initialAudio), 25000); await waitForSample(() => read(viewers[1]!), (s) => s.width > 0 && s.frames >= beforeSource.frames+20 && s.audio===Number(!initialAudio), 25000);
+33
View File
@@ -78,6 +78,7 @@ interface GateResult {
liveQualityChanged: boolean; liveQualityChanged: boolean;
pausedQualityChanged: boolean; pausedQualityChanged: boolean;
backgroundProfileRecovery: boolean | null; backgroundProfileRecovery: boolean | null;
sourceWindowRecovery: boolean | null;
mediaObjectPreserved: boolean; mediaObjectPreserved: boolean;
nativeSourceChanged: boolean; nativeSourceChanged: boolean;
nativeQualityEvidence: boolean; nativeQualityEvidence: boolean;
@@ -641,6 +642,7 @@ async function main(): Promise<void> {
liveQualityChanged: false, liveQualityChanged: false,
pausedQualityChanged: false, pausedQualityChanged: false,
backgroundProfileRecovery: mode === "local" && sourceKind === "window" ? false : null, backgroundProfileRecovery: mode === "local" && sourceKind === "window" ? false : null,
sourceWindowRecovery: mode === "local" && sourceKind === "window" ? false : null,
mediaObjectPreserved: false, mediaObjectPreserved: false,
nativeSourceChanged: false, nativeSourceChanged: false,
nativeQualityEvidence: false, nativeQualityEvidence: false,
@@ -1242,6 +1244,36 @@ async function main(): Promise<void> {
), Boolean, 5_000); ), Boolean, 5_000);
} }
result.backgroundProfileRecovery = true; result.backgroundProfileRecovery = true;
// WGC must keep the share through size changes and a temporarily quiet
// minimized window. This is a window-transition check, not a game test.
for (const bounds of [
{ width: 960, height: 650 },
{ windowState: "fullscreen" },
{ windowState: "normal" },
{ windowState: "minimized" },
]) {
stage = "native-window-" + (bounds.windowState ?? "resize");
await sourceCdp!.call("Browser.setWindowBounds", {
windowId, bounds,
}, undefined, Date.now() + 5_000);
if (bounds.windowState === "minimized") {
await new Promise((resolveWait) => setTimeout(resolveWait, 5_000));
await sourceCdp!.call("Browser.setWindowBounds", {
windowId, bounds: { windowState: "normal" },
}, undefined, Date.now() + 5_000);
}
const before = await evaluate<number>(cdp, viewer,
"document.querySelector('video')?.getVideoPlaybackQuality().totalVideoFrames ?? 0", Date.now() + 5_000);
await waitForValue((deadline) => evaluate<boolean>(cdp!, viewer,
`(() => {
const video = document.querySelector('video');
return Boolean(video && video.srcObject === window.__piikGateMedia &&
video.getVideoPlaybackQuality().totalVideoFrames >= ${before + 30});
})()`, deadline,
), Boolean, 20_000);
await assertVideoCodec(cdp, viewer, actualCodec);
}
result.sourceWindowRecovery = true;
} }
stage = "native-source-picker"; stage = "native-source-picker";
await waitForValue( await waitForValue(
@@ -1481,6 +1513,7 @@ async function main(): Promise<void> {
: result.viewerConnected && result.viewerFrames >= 30 && : result.viewerConnected && result.viewerFrames >= 30 &&
result.qualityControlsEnabled && result.liveQualityChanged && result.qualityControlsEnabled && result.liveQualityChanged &&
result.backgroundProfileRecovery !== false && result.backgroundProfileRecovery !== false &&
result.sourceWindowRecovery !== false &&
result.livePresetChanges === 2 && result.livePresetChanges === 2 &&
result.codecPreserved && result.codecPreserved &&
result.pausedQualityChanged && result.mediaObjectPreserved && result.pausedQualityChanged && result.mediaObjectPreserved &&
+4 -3
View File
@@ -12,6 +12,7 @@ const single = settings.get('single') === '1', late = settings.get('late') === '
const nativeSource = settings.get('nativeSource') === '1', background = settings.get('background') === '1'; const nativeSource = settings.get('nativeSource') === '1', background = settings.get('background') === '1';
const lifecycle = settings.get('lifecycle') === '1', shapedRate = Number(settings.get('rate') || 120000); const lifecycle = settings.get('lifecycle') === '1', shapedRate = Number(settings.get('rate') || 120000);
const relay = settings.get('relay') === '1'; const relay = settings.get('relay') === '1';
const shortPulse = settings.get('shortPulse') === '1';
const selectedProfile = () => ({ resolution: high ? '1080p' : '480p', maxFramerate: 30, const selectedProfile = () => ({ resolution: high ? '1080p' : '480p', maxFramerate: 30,
maxBitrate: high ? 5000000 : 1700000, maxBitrate: high ? 5000000 : 1700000,
degradationPreference: settings.get('preference') || (high ? 'maintain-resolution' : 'balanced'), degradationPreference: settings.get('preference') || (high ? 'maintain-resolution' : 'balanced'),
@@ -282,7 +283,7 @@ async function run() {
extensions: e.send.localDescription?.sdp.match(/^a=extmap:.*$/gm), extensions: e.send.localDescription?.sdp.match(/^a=extmap:.*$/gm),
parameters: e.sender.getParameters().encodings })); parameters: e.sender.getParameters().encodings }));
const snapshots = [await snapshot()]; const snapshots = [await snapshot()];
phase = 'healthy'; await wait(6000); snapshots.push(await snapshot()); phase = 'healthy'; await wait(shortPulse ? 45000 : 6000); snapshots.push(await snapshot());
if (lifecycle) { if (lifecycle) {
let appliedProfile = selectedProfile(); let appliedProfile = selectedProfile();
const apply = async profile => { const apply = async profile => {
@@ -341,7 +342,7 @@ async function run() {
} }
if (network) { if (network) {
await fetch('/shaper?rate=' + shapedRate); await fetch('/shaper?rate=' + shapedRate);
phase = 'limited'; await wait(14000); snapshots.push(await snapshot()); phase = 'limited'; await wait(shortPulse ? 1000 : 14000); snapshots.push(await snapshot());
await fetch('/shaper?rate=0'); await fetch('/shaper?rate=0');
// --auto extends recovery observation; adaptation always belongs to the product. // --auto extends recovery observation; adaptation always belongs to the product.
phase = 'released'; await wait(automatic ? 40000 : 10000); snapshots.push(await snapshot()); phase = 'released'; await wait(automatic ? 40000 : 10000); snapshots.push(await snapshot());
@@ -350,7 +351,7 @@ async function run() {
audioMonitors: audioMonitors.map(({ role, pulses, attached }) => ({ role, pulses, attached })), audioMonitors: audioMonitors.map(({ role, pulses, attached }) => ({ role, pulses, attached })),
monitors: monitors.map(({ role, samples }) => ({ role, samples })) }; monitors: monitors.map(({ role, samples }) => ({ role, samples })) };
} }
const configuration = { mode, codec, product: true, network, automatic, av, high, single, late, const configuration = { mode, codec, product: true, network, automatic, av, high, single, late, shortPulse,
background, nativeSource, lifecycle, relay, shapedRate, profile: selectedProfile() }; background, nativeSource, lifecycle, relay, shapedRate, profile: selectedProfile() };
try { try {
window.probeResult = { ...configuration, ...await run(), controls, visibility, productTrace, errors }; window.probeResult = { ...configuration, ...await run(), controls, visibility, productTrace, errors };
+6 -4
View File
@@ -9,7 +9,7 @@ import { createPoolShaper } from './browser-pool-shaper.mjs';
const mode = process.argv[2] || 'carrier', args = process.argv.slice(3); const mode = process.argv[2] || 'carrier', args = process.argv.slice(3);
const flags = new Set(['--product', '--1080', '--h264', '--single', '--av', '--late', '--background', const flags = new Set(['--product', '--1080', '--h264', '--single', '--av', '--late', '--background',
'--native-source', '--lifecycle', '--network', '--auto', '--relay', '--debug']); '--native-source', '--lifecycle', '--network', '--auto', '--relay', '--debug', '--short-pulse']);
if (!['ordinary', 'carrier'].includes(mode)) throw Error('Expected ordinary|carrier'); if (!['ordinary', 'carrier'].includes(mode)) throw Error('Expected ordinary|carrier');
for (const arg of args) if (!flags.has(arg) && !arg.startsWith('--rate=') && !arg.startsWith('--preference=')) for (const arg of args) if (!flags.has(arg) && !arg.startsWith('--rate=') && !arg.startsWith('--preference='))
throw Error('Unsupported option: ' + arg); throw Error('Unsupported option: ' + arg);
@@ -18,12 +18,14 @@ const automatic = args.includes('--auto'), high = args.includes('--1080'), av =
const single = args.includes('--single'), late = args.includes('--late'), background = args.includes('--background'); const single = args.includes('--single'), late = args.includes('--late'), background = args.includes('--background');
const nativeSource = background || args.includes('--native-source'), lifecycle = args.includes('--lifecycle'); const nativeSource = background || args.includes('--native-source'), lifecycle = args.includes('--lifecycle');
const relay = args.includes('--relay'); const relay = args.includes('--relay');
const shortPulse = args.includes('--short-pulse');
const rate = Number(args.find(arg => arg.startsWith('--rate='))?.split('=')[1] || 120000); const rate = Number(args.find(arg => arg.startsWith('--rate='))?.split('=')[1] || 120000);
const preference = args.find(arg => arg.startsWith('--preference='))?.split('=')[1] || (high ? 'maintain-resolution' : 'balanced'); const preference = args.find(arg => arg.startsWith('--preference='))?.split('=')[1] || (high ? 'maintain-resolution' : 'balanced');
if (!['balanced', 'maintain-resolution', 'maintain-framerate'].includes(preference)) throw Error('Unsupported degradation preference'); if (!['balanced', 'maintain-resolution', 'maintain-framerate'].includes(preference)) throw Error('Unsupported degradation preference');
if (!Number.isInteger(rate) || rate <= 0) throw Error('Rate must be a positive integer in bps'); if (!Number.isInteger(rate) || rate <= 0) throw Error('Rate must be a positive integer in bps');
if (lifecycle && (nativeSource || network || relay)) throw Error('Lifecycle uses direct canvas capture without network shaping'); if (lifecycle && (nativeSource || network || relay)) throw Error('Lifecycle uses direct canvas capture without network shaping');
if (single && late) throw Error('Single-viewer control cannot also late-join B'); if (single && late) throw Error('Single-viewer control cannot also late-join B');
if (shortPulse && (!network || !automatic)) throw Error('Short pulse requires --network --auto');
const root = resolve(import.meta.dirname, '..'); const root = resolve(import.meta.dirname, '..');
const bundle = await build({ entryPoints: [join(root, 'scripts/browser-local-pool-page.js')], const bundle = await build({ entryPoints: [join(root, 'scripts/browser-local-pool-page.js')],
bundle: true, write: false, format: 'esm', platform: 'browser', logLevel: 'error' }); bundle: true, write: false, format: 'esm', platform: 'browser', logLevel: 'error' });
@@ -71,7 +73,7 @@ const chrome = await launchChrome(process.env.CHROME_PATH || 'C:/Program Files/G
...(background ? [] : ['--disable-background-timer-throttling', '--disable-renderer-backgrounding']), ...(background ? [] : ['--disable-background-timer-throttling', '--disable-renderer-backgrounding']),
...(nativeSource ? ['--use-fake-device-for-media-stream', '--use-fake-ui-for-media-stream'] : [])]); ...(nativeSource ? ['--use-fake-device-for-media-stream', '--use-fake-ui-for-media-stream'] : [])]);
chrome.stdout.resume(); chrome.stderr.resume(); chrome.stdout.resume(); chrome.stderr.resume();
let result: Record<string, unknown> = { mode, codec, product: true, network, automatic, high, av, single, late, let result: Record<string, unknown> = { mode, codec, product: true, network, automatic, high, av, single, late, shortPulse,
background, nativeSource, lifecycle, relay, shapedRate: rate, preference }; background, nativeSource, lifecycle, relay, shapedRate: rate, preference };
try { try {
const version = await waitForVersion(debugPort, chrome); const version = await waitForVersion(debugPort, chrome);
@@ -79,7 +81,7 @@ try {
page = await createPage(cdp, 'about:blank'); page = await createPage(cdp, 'about:blank');
const query = new URLSearchParams({ mode, codec, rate: String(rate), preference }); const query = new URLSearchParams({ mode, codec, rate: String(rate), preference });
if (process.argv.includes('--debug')) query.set('debug', '1'); if (process.argv.includes('--debug')) query.set('debug', '1');
for (const [key, value] of Object.entries({ network, automatic, high, av, single, late, background, nativeSource, lifecycle, relay })) for (const [key, value] of Object.entries({ network, automatic, high, av, single, late, background, nativeSource, lifecycle, relay, shortPulse }))
query.set(key, value ? '1' : '0'); query.set(key, value ? '1' : '0');
await cdp.call('Page.navigate', { url: 'http://127.0.0.1:' + port + '/?' + query }, page.sessionId, Date.now() + 5000); await cdp.call('Page.navigate', { url: 'http://127.0.0.1:' + port + '/?' + query }, page.sessionId, Date.now() + 5000);
await waitForSample(() => evaluate<boolean>(cdp!, page!, 'window.probeDone===true', Date.now() + 2000), await waitForSample(() => evaluate<boolean>(cdp!, page!, 'window.probeDone===true', Date.now() + 2000),
@@ -96,7 +98,7 @@ try {
result.probeSha256 = createHash('sha256').update(body).digest('hex'); result.probeSha256 = createHash('sha256').update(body).digest('hex');
result.cleanup = clean; result.cleanup = clean;
await mkdir(join(root, 'build/browser-local-pool'), { recursive: true }); await mkdir(join(root, 'build/browser-local-pool'), { recursive: true });
const name = [mode, codec.toLowerCase(), network && 'network', automatic && 'auto', high && '1080', const name = [mode, codec.toLowerCase(), network && 'network', automatic && 'auto', shortPulse && 'pulse', high && '1080',
single && 'single', late && 'late', background && 'background', lifecycle && 'lifecycle', relay && 'relay', Date.now()].filter(Boolean).join('-'); single && 'single', late && 'late', background && 'background', lifecycle && 'lifecycle', relay && 'relay', Date.now()].filter(Boolean).join('-');
const output = join(root, 'build/browser-local-pool', name + '.json'); const output = join(root, 'build/browser-local-pool', name + '.json');
await writeFile(output, JSON.stringify(result, null, 2) + '\n'); await writeFile(output, JSON.stringify(result, null, 2) + '\n');
+1 -1
View File
@@ -5,7 +5,7 @@ for (const filename of process.argv.slice(2)) {
result.cleanup && Object.values(result.cleanup).every(Boolean); result.cleanup && Object.values(result.cleanup).every(Boolean);
const configuration = { file: filename, probeSha256: result.probeSha256, mode: result.mode, codec: result.codec, const configuration = { file: filename, probeSha256: result.probeSha256, mode: result.mode, codec: result.codec,
product: result.product, high: result.high, single: result.single, late: result.late, network: result.network, product: result.product, high: result.high, single: result.single, late: result.late, network: result.network,
automatic: result.automatic, av: result.av, background: result.background, nativeSource: result.nativeSource, automatic: result.automatic, shortPulse: result.shortPulse, av: result.av, background: result.background, nativeSource: result.nativeSource,
lifecycle: result.lifecycle, relay: result.relay, shapedRate: result.shapedRate, profile: result.profile, lifecycle: result.lifecycle, relay: result.relay, shapedRate: result.shapedRate, profile: result.profile,
completed: !!completed, error: result.error, errors: result.errors, cleanup: result.cleanup }; completed: !!completed, error: result.error, errors: result.errors, cleanup: result.cleanup };
if (!completed) process.exitCode = 1; if (!completed) process.exitCode = 1;
+9 -1
View File
@@ -26,11 +26,19 @@ export class ApiError extends Error {
} }
async function responseBody(response: Response): Promise<unknown> { async function responseBody(response: Response): Promise<unknown> {
let body: unknown;
try { try {
return await response.json(); body = await response.json();
} catch { } catch {
throw new ApiError(say("host.err.serverError"), response.status); throw new ApiError(say("host.err.serverError"), response.status);
} }
if (
response.status === 403 && body && typeof body === "object" &&
"error" in body && body.error === "Origin not allowed"
) {
throw new ApiError(say("api.originNotAllowed"), response.status);
}
return body;
} }
function parseSiteAccessStatus(value: unknown): SiteAccessStatus { function parseSiteAccessStatus(value: unknown): SiteAccessStatus {
+1
View File
@@ -385,6 +385,7 @@ export const en: Record<CopyKey, string> = {
"gate.connectFailed": "Cannot reach the site access service; try again", "gate.connectFailed": "Cannot reach the site access service; try again",
"gate.expired": "Site access expired; verify again", "gate.expired": "Site access expired; verify again",
"gate.serviceUnavailable": "Site access service unavailable ({status})", "gate.serviceUnavailable": "Site access service unavailable ({status})",
"api.originNotAllowed": "This address is not allowed by the server (403). Ask the site administrator to check PUBLIC_BASE_URL and ALLOWED_ORIGINS.",
"host.notStarted": "Not started", "host.notStarted": "Not started",
"host.startCancelled": "Start cancelled", "host.startCancelled": "Start cancelled",
+1
View File
@@ -385,6 +385,7 @@ export const zh = {
"gate.connectFailed": "无法连接站点访问服务,请重试", "gate.connectFailed": "无法连接站点访问服务,请重试",
"gate.expired": "站点访问已失效,请重新验证", "gate.expired": "站点访问已失效,请重新验证",
"gate.serviceUnavailable": "站点验证服务暂时不可用 ({status})", "gate.serviceUnavailable": "站点验证服务暂时不可用 ({status})",
"api.originNotAllowed": "服务器未允许当前网址 (403)。请联系站点管理员检查 PUBLIC_BASE_URL 和 ALLOWED_ORIGINS。",
"host.notStarted": "尚未开始", "host.notStarted": "尚未开始",
"host.startCancelled": "启动已取消", "host.startCancelled": "启动已取消",
+3 -1
View File
@@ -63,7 +63,9 @@ export class BrowserEncodingOutput {
if (!context) throw new Error("Browser carrier canvas unavailable"); if (!context) throw new Error("Browser carrier canvas unavailable");
this.clockContext = context; this.clockContext = context;
this.track = this.clock.captureStream(0).getVideoTracks()[0] as CanvasCaptureMediaStreamTrack; this.track = this.clock.captureStream(0).getVideoTracks()[0] as CanvasCaptureMediaStreamTrack;
this.track.contentHint = "motion"; // This synthetic clock needs WebRTC's screen-content ALR probing when the
// real producer sends little data. The producer owns picture adaptation.
this.track.contentHint = "detail";
const streams = encodedStreams(sender); const streams = encodedStreams(sender);
this.writer = streams.writable.getWriter(); this.writer = streams.writable.getWriter();
void streams.readable.pipeTo(new WritableStream({ write: (frame) => this.receive(frame) }), void streams.readable.pipeTo(new WritableStream({ write: (frame) => this.receive(frame) }),
+5 -3
View File
@@ -217,6 +217,7 @@ export class BrowserEncodingPool {
member.encoded.select(pending.group.producer.id, () => pending.group.producer.requestKey(), () => { member.encoded.select(pending.group.producer.id, () => pending.group.producer.requestKey(), () => {
if (member.disposed || member.pending !== pending) return; if (member.disposed || member.pending !== pending) return;
member.current = pending.group; member.current = pending.group;
pending.group.producer.beginOutput();
member.pending = undefined; member.pending = undefined;
member.encoderStats = createStatsAccumulator(); member.previousOutput = undefined; member.encoderStats = createStatsAccumulator(); member.previousOutput = undefined;
member.carrier = true; member.carrier = true;
@@ -247,9 +248,10 @@ export class BrowserEncodingPool {
} }
continue; continue;
} }
// An unshared pipeline already has the right owner: let its native // A short output-rate spike is not new demand. Keep the rate owner and
// encoder adapt to the updated budget instead of recreating it. // let its native encoder adapt unless another child needs a higher rate.
if (current && this.compatible(member, current) && !this.references(current).some((other) => other !== member)) continue; if (current && this.compatible(member, current) && !this.references(current).some((other) =>
this.compatible(other, current) && other.budget !== undefined && other.budget > member.budget!)) continue;
const candidates = [...this.groups].filter((group) => this.compatible(member, group)); const candidates = [...this.groups].filter((group) => this.compatible(member, group));
let target = candidates.find((group) => group !== current && fits(group.output, member.budget) && let target = candidates.find((group) => group !== current && fits(group.output, member.budget) &&
(!current || group.budget === member.budget)); (!current || group.budget === member.budget));
+14 -3
View File
@@ -1,7 +1,6 @@
import { createOpaqueId } from "../lib/opaque-id"; import { createOpaqueId } from "../lib/opaque-id";
import { debugError, debugEvent } from "../lib/debug"; import { debugError, debugEvent } from "../lib/debug";
import { debugRtcFailure, debugRtcStats, debugTrack, observeDebugConnection } from "../lib/debug-webrtc"; import { debugRtcFailure, debugRtcStats, debugTrack, observeDebugConnection } from "../lib/debug-webrtc";
import { maxEncodedVideoFrames } from "../webrtc/stats";
import { addRemoteIceCandidate } from "../webrtc/nat-prediction"; import { addRemoteIceCandidate } from "../webrtc/nat-prediction";
import { encodedStreams } from "./browser-encoding-output"; import { encodedStreams } from "./browser-encoding-output";
import { import {
@@ -28,6 +27,7 @@ export class BrowserEncodingProducer {
private paused: boolean; private paused: boolean;
private budget: number; private budget: number;
private startupPending: boolean; private startupPending: boolean;
private startupFrames: number | undefined;
private startupTimer: ReturnType<typeof setTimeout> | undefined; private startupTimer: ReturnType<typeof setTimeout> | undefined;
private applied: { capture: QualityProfile; sender: QualityProfile } | null = null; private applied: { capture: QualityProfile; sender: QualityProfile } | null = null;
@@ -67,6 +67,9 @@ export class BrowserEncodingProducer {
this.streamWriter = writer; this.streamWriter = writer;
void streams.readable.pipeTo(new WritableStream({ write: async (frame) => { void streams.readable.pipeTo(new WritableStream({ write: async (frame) => {
if (this.disposed) return; if (this.disposed) return;
if (this.startupPending && this.startupFrames !== undefined && !this.paused && frame.data.byteLength > 0) {
this.startupFrames = Math.min(STARTUP_VIDEO_ENCODED_FRAMES, this.startupFrames + 1);
}
// Outputs clone synchronously before the local decoder consumes the original. // Outputs clone synchronously before the local decoder consumes the original.
this.onFrame(frame); this.onFrame(frame);
await writer.write(frame); await writer.write(frame);
@@ -142,7 +145,15 @@ export class BrowserEncodingProducer {
const request = sender.setParameters as (parameters: RTCRtpSendParameters, const request = sender.setParameters as (parameters: RTCRtpSendParameters,
options: { encodingOptions: Array<{ keyFrame: boolean }> }) => Promise<void>; options: { encodingOptions: Array<{ keyFrame: boolean }> }) => Promise<void>;
await request.call(sender, sender.getParameters(), { encodingOptions: [{ keyFrame: true }] }); await request.call(sender, sender.getParameters(), { encodingOptions: [{ keyFrame: true }] });
}); // Producer failure falls back to ordinary encoding on the same connection.
// Its retired keyframe requests must not fail that outgoing connection.
}).catch(this.fail);
}
beginOutput(): void {
// Local warmup cannot train an outgoing connection's allocation. Start the
// existing protection only after a child actually writes this producer.
this.startupFrames ??= 0;
} }
async report(): Promise<RTCStatsReport> { async report(): Promise<RTCStatsReport> {
@@ -151,7 +162,7 @@ export class BrowserEncodingProducer {
const report = await connection.getStats().catch((error) => { debugRtcFailure(connection, error); throw error; }); const report = await connection.getStats().catch((error) => { debugRtcFailure(connection, error); throw error; });
debugRtcStats(connection, report); debugRtcStats(connection, report);
if (!this.disposed && this.startupPending && if (!this.disposed && this.startupPending &&
(maxEncodedVideoFrames(report, this.input!.id) ?? 0) >= STARTUP_VIDEO_ENCODED_FRAMES) { (this.startupFrames ?? 0) >= STARTUP_VIDEO_ENCODED_FRAMES) {
this.startupPending = false; this.startupPending = false;
void this.serialize(() => this.applyCurrent()).catch(this.fail); void this.serialize(() => this.applyCurrent()).catch(this.fail);
} }
+19 -42
View File
@@ -161,30 +161,11 @@ export class HostSfuRoute {
this.resyncing = true; this.resyncing = true;
this.route.reset(); this.route.reset();
this.recovery = null; this.recovery = null;
const pending = this.pending; const closing = this.retirePublishers();
const active = this.active;
this.pending = null;
this.active = null;
if (pending) {
pending.failed = true;
}
if (active) {
active.failed = true;
}
const accepted = this.route.accept(update); const accepted = this.route.accept(update);
this.events.reconcileChildren([]); this.events.reconcileChildren([]);
await this.queueTransition(async () => { await this.queueTransition(() => closing);
if (active?.active) {
await active.publisher.deactivate().catch(() => false);
}
if (active) {
await disconnectPublisher(active.publisher);
}
if (pending && pending !== active) {
await disconnectPublisher(pending.publisher);
}
});
if (this.closed || this.resyncGeneration !== resyncGeneration) { if (this.closed || this.resyncGeneration !== resyncGeneration) {
return "stale"; return "stale";
@@ -403,27 +384,23 @@ export class HostSfuRoute {
this.resyncing = false; this.resyncing = false;
this.route.reset(); this.route.reset();
this.recovery = null; this.recovery = null;
await this.queueTransition(async () => { const closing = this.retirePublishers();
const pending = this.pending; await this.queueTransition(() => closing);
const active = this.active; }
this.pending = null;
this.active = null; private async retirePublishers(): Promise<void> {
if (pending) { const pending = this.pending;
pending.failed = true; const active = this.active;
} this.pending = null;
if (active) { this.active = null;
active.failed = true; if (pending) pending.failed = true;
} if (active) active.failed = true;
if (active?.active) { // Cancel before joining the transition queue: its current operation may
await active.publisher.deactivate().catch(() => false); // itself be waiting for one of these publishers to finish preparation.
} await Promise.all([
await Promise.all([ pending ? disconnectPublisher(pending.publisher) : undefined,
pending ? disconnectPublisher(pending.publisher) : undefined, active && active !== pending ? this.disconnectRetiredPublisher(active) : undefined,
active && active !== pending ]);
? disconnectPublisher(active.publisher)
: undefined,
]);
});
} }
private queueActivation( private queueActivation(
+3 -2
View File
@@ -2485,19 +2485,20 @@ export function HostPage({
); );
} else { } else {
setDetails(captureDetails(captured)); setDetails(captureDetails(captured));
videoCodecRef.current = await resolveStreamVideoCodec(captured); const codec = await resolveStreamVideoCodec(captured);
if (isCurrentShare(generation, shareGeneration)) { if (isCurrentShare(generation, shareGeneration)) {
videoCodecRef.current = codec;
await startBrowserNativeIngress(generation, shareGeneration, captured); await startBrowserNativeIngress(generation, shareGeneration, captured);
nativeStarted = nativeMediaIngressRef.current !== null; nativeStarted = nativeMediaIngressRef.current !== null;
} }
} }
setResolvedVideoCodec(videoCodecRef.current.primary);
} }
if (!isCurrentShare(generation, shareGeneration)) { if (!isCurrentShare(generation, shareGeneration)) {
captured?.getTracks().forEach((track) => track.stop()); captured?.getTracks().forEach((track) => track.stop());
if (nativeStarted) disposeNativeShare(shareGeneration); if (nativeStarted) disposeNativeShare(shareGeneration);
return; return;
} }
if (captured) setResolvedVideoCodec(videoCodecRef.current.primary);
let createdRoom: HostRoomState | null = null; let createdRoom: HostRoomState | null = null;
let claimedRoom = false; let claimedRoom = false;
+13 -7
View File
@@ -1,5 +1,6 @@
import { observeDebugConnection } from "../lib/debug-webrtc"; import { observeDebugConnection } from "../lib/debug-webrtc";
import { addRemoteIceCandidate } from "../webrtc/nat-prediction"; import { addRemoteIceCandidate } from "../webrtc/nat-prediction";
import { waitForConnectionOperation } from "../webrtc/connection-operation";
import type { import type {
ServerMessage, ServerMessage,
SfuMedia, SfuMedia,
@@ -27,7 +28,8 @@ export class SfuPeer {
SfuSignalMessage, SfuSignalMessage,
"kind" | "description" | "media" "kind" | "description" | "media"
> | null = null; > | null = null;
private closed = false; private readonly lifetime = new AbortController();
private get closed(): boolean { return this.lifetime.signal.aborted; }
private signalTail: Promise<void> = Promise.resolve(); private signalTail: Promise<void> = Promise.resolve();
constructor( constructor(
@@ -86,7 +88,7 @@ export class SfuPeer {
media?: SfuMedia, media?: SfuMedia,
): Promise<void> { ): Promise<void> {
this.descriptionSent = false; this.descriptionSent = false;
await this.pc.setLocalDescription(description); await this.waitForOperation(() => this.pc.setLocalDescription(description));
if (this.closed) return; if (this.closed) return;
const local = this.pc.localDescription; const local = this.pc.localDescription;
if (!local || (local.type !== "offer" && local.type !== "answer")) { if (!local || (local.type !== "offer" && local.type !== "answer")) {
@@ -129,14 +131,14 @@ export class SfuPeer {
const work = async (): Promise<void> => { const work = async (): Promise<void> => {
if (this.closed) return; if (this.closed) return;
if (message.kind === "layers" && message.activeCount !== undefined) { if (message.kind === "layers" && message.activeCount !== undefined) {
await onLayers?.(message.activeCount); await this.waitForOperation(() => onLayers?.(message.activeCount!) ?? Promise.resolve());
} else if (message.kind === "description" && message.description) { } else if (message.kind === "description" && message.description) {
await onDescription(message.description); await this.waitForOperation(() => onDescription(message.description!));
for (const candidate of this.remoteCandidates.splice(0)) for (const candidate of this.remoteCandidates.splice(0))
await addRemoteIceCandidate(this.pc, candidate); await this.waitForOperation(() => addRemoteIceCandidate(this.pc, candidate));
} else if (message.kind === "candidate" && message.candidate) { } else if (message.kind === "candidate" && message.candidate) {
if (this.pc.remoteDescription) if (this.pc.remoteDescription)
await addRemoteIceCandidate(this.pc, message.candidate); await this.waitForOperation(() => addRemoteIceCandidate(this.pc, message.candidate!));
else if (this.remoteCandidates.length < 64) else if (this.remoteCandidates.length < 64)
this.remoteCandidates.push(message.candidate); this.remoteCandidates.push(message.candidate);
else throw new Error("Too many SFU ICE candidates"); else throw new Error("Too many SFU ICE candidates");
@@ -147,9 +149,13 @@ export class SfuPeer {
return next; return next;
} }
waitForOperation<T>(operation: () => Promise<T>): Promise<T> {
return waitForConnectionOperation(this.lifetime.signal, operation);
}
close(): void { close(): void {
if (this.closed) return; if (this.closed) return;
this.closed = true; this.lifetime.abort();
this.pc.onicecandidate = null; this.pc.onicecandidate = null;
this.pc.onconnectionstatechange = null; this.pc.onconnectionstatechange = null;
this.pc.ontrack = null; this.pc.ontrack = null;
+29 -24
View File
@@ -107,7 +107,7 @@ export class SfuPublisher {
async (description) => { async (description) => {
if (description.type !== "answer") if (description.type !== "answer")
throw new Error("SFU publisher expected an answer"); throw new Error("SFU publisher expected an answer");
await peer.pc.setRemoteDescription(description); await peer.waitForOperation(() => peer.pc.setRemoteDescription(description));
}, },
async (activeCount) => { async (activeCount) => {
await this.enqueue(async () => { await this.enqueue(async () => {
@@ -120,7 +120,7 @@ export class SfuPublisher {
parameters.encodings.forEach((encoding, index) => { parameters.encodings.forEach((encoding, index) => {
encoding.active = index < activeCount; encoding.active = index < activeCount;
}); });
await sender.setParameters(parameters); await peer.waitForOperation(() => sender.setParameters(parameters));
return this.peer === peer; return this.peer === peer;
}); });
}, },
@@ -147,7 +147,7 @@ export class SfuPublisher {
this.failureStage = "source"; this.failureStage = "source";
const sourceVideo = requiredVideo(stream); const sourceVideo = requiredVideo(stream);
video = this.ownTrack(cloneSenderVideoTrack(sourceVideo)); video = this.ownTrack(cloneSenderVideoTrack(sourceVideo));
await applyVideoCaptureProfile(video, profile); await peer.waitForOperation(() => applyVideoCaptureProfile(video!, profile));
if (this.peer !== peer) { if (this.peer !== peer) {
this.releaseTrack(video); this.releaseTrack(video);
return false; return false;
@@ -193,7 +193,7 @@ export class SfuPublisher {
await this.configure(profile); await this.configure(profile);
if (this.peer !== peer) return false; if (this.peer !== peer) return false;
await peer.sendDescription( await peer.sendDescription(
await peer.pc.createOffer(), await peer.waitForOperation(() => peer.pc.createOffer()),
this.media(profile), this.media(profile),
); );
if (this.peer !== peer) return false; if (this.peer !== peer) return false;
@@ -201,11 +201,12 @@ export class SfuPublisher {
this.startStats(); this.startStats();
return true; return true;
} catch (error) { } catch (error) {
if (video && this.video !== video) this.releaseTrack(video);
if (this.peer !== peer) return false;
debugError("webrtc", "sfu-activate-failed", error, { debugError("webrtc", "sfu-activate-failed", error, {
stage: this.failureStage ?? "connect", stage: this.failureStage ?? "connect",
}); });
if (video && this.video !== video) this.releaseTrack(video); this.fail(this.failureStage ?? "connect");
if (this.peer === peer) this.fail(this.failureStage ?? "connect");
throw error; throw error;
} }
}); });
@@ -238,15 +239,15 @@ export class SfuPublisher {
const previousBaseline = this.startupFramesBaseline; const previousBaseline = this.startupFramesBaseline;
if (videoChanged) this.resetStats(); if (videoChanged) this.resetStats();
try { try {
if (videoChanged) await applyVideoCaptureProfile(nextVideo, profile); if (videoChanged) await peer.waitForOperation(() => applyVideoCaptureProfile(nextVideo, profile));
if (this.peer !== peer) return false; if (this.peer !== peer) return false;
nextVideo.enabled = !this.paused; nextVideo.enabled = !this.paused;
if (nextAudio) { if (nextAudio) {
nextAudio.enabled = !this.paused; nextAudio.enabled = !this.paused;
nextAudio.contentHint = "music"; nextAudio.contentHint = "music";
} }
if (videoChanged) await videoSender.replaceTrack(nextVideo); if (videoChanged) await peer.waitForOperation(() => videoSender.replaceTrack(nextVideo));
await audioSender.replaceTrack(nextAudio); await peer.waitForOperation(() => audioSender.replaceTrack(nextAudio));
if (this.peer !== peer) return false; if (this.peer !== peer) return false;
this.video = nextVideo; this.video = nextVideo;
this.audio = nextAudio; this.audio = nextAudio;
@@ -265,8 +266,8 @@ export class SfuPublisher {
this.setPaused(this.paused); this.setPaused(this.paused);
return true; return true;
} catch (error) { } catch (error) {
debugError("webrtc", "sfu-source-failed", error);
if (this.peer !== peer) return false; if (this.peer !== peer) return false;
debugError("webrtc", "sfu-source-failed", error);
this.video = previousVideo; this.video = previousVideo;
this.audio = previousAudio; this.audio = previousAudio;
if (videoChanged) { if (videoChanged) {
@@ -274,9 +275,10 @@ export class SfuPublisher {
this.startupFramesBaseline = previousBaseline; this.startupFramesBaseline = previousBaseline;
} }
try { try {
if (videoChanged) await videoSender.replaceTrack(previousVideo); if (videoChanged) await peer.waitForOperation(() => videoSender.replaceTrack(previousVideo));
await audioSender.replaceTrack(previousAudio); await peer.waitForOperation(() => audioSender.replaceTrack(previousAudio));
await this.configure(profile, videoChanged); await this.configure(profile, videoChanged);
if (this.peer !== peer) return false;
this.videoWarning = { key: "host.fail.sfuSwitch" }; this.videoWarning = { key: "host.fail.sfuSwitch" };
return false; return false;
} catch { } catch {
@@ -310,7 +312,7 @@ export class SfuPublisher {
.degradationPreference; .degradationPreference;
if (configureVideo) this.resetStats(); if (configureVideo) this.resetStats();
try { try {
if (videoChanged) await applyVideoCaptureProfile(video, profile); if (videoChanged) await peer.waitForOperation(() => applyVideoCaptureProfile(video, profile));
const result = await this.configure(profile, configureVideo); const result = await this.configure(profile, configureVideo);
if (this.peer !== peer) return false; if (this.peer !== peer) return false;
if (!peer.send({ kind: "media", media: this.media(profile) })) if (!peer.send({ kind: "media", media: this.media(profile) }))
@@ -318,11 +320,12 @@ export class SfuPublisher {
this.profile = profile; this.profile = profile;
return result; return result;
} catch (error) { } catch (error) {
debugError("webrtc", "sfu-profile-failed", error, { requested: profile });
if (this.peer !== peer) return false; if (this.peer !== peer) return false;
debugError("webrtc", "sfu-profile-failed", error, { requested: profile });
try { try {
if (videoChanged) await applyVideoCaptureProfile(video, previous); if (videoChanged) await peer.waitForOperation(() => applyVideoCaptureProfile(video, previous));
await this.configure(previous, configureVideo); await this.configure(previous, configureVideo);
if (this.peer !== peer) return false;
this.videoWarning = { key: "host.fail.sfuParams" }; this.videoWarning = { key: "host.fail.sfuParams" };
return false; return false;
} catch { } catch {
@@ -390,7 +393,7 @@ export class SfuPublisher {
return false; return false;
try { try {
await peer.sendDescription( await peer.sendDescription(
await peer.pc.createOffer({ iceRestart: true }), await peer.waitForOperation(() => peer.pc.createOffer({ iceRestart: true })),
this.media(this.profile), this.media(this.profile),
); );
return true; return true;
@@ -478,29 +481,31 @@ export class SfuPublisher {
parameters.encodings.forEach((encoding, index) => parameters.encodings.forEach((encoding, index) =>
Object.assign(encoding, encodings[index]), Object.assign(encoding, encodings[index]),
); );
await sender.setParameters(parameters); await peer.waitForOperation(() => sender.setParameters(parameters));
if (this.peer !== peer) return false; if (this.peer !== peer) return false;
const readback = await configureVideoSender( const readback = await peer.waitForOperation(() => configureVideoSender(
sender, sender,
this.startupPending ? startupVideoProfile(profile) : profile, this.startupPending ? startupVideoProfile(profile) : profile,
); ));
if (this.peer !== peer) return false; if (this.peer !== peer) return false;
this.senderParameters = readback; this.senderParameters = readback;
this.videoWarning = senderParameterWarning(readback); this.videoWarning = senderParameterWarning(readback);
} }
if (!this.audio || !this.audioSender) { const audioSender = this.audioSender;
if (!this.audio || !audioSender) {
this.audioWarning = null; this.audioWarning = null;
return true; return true;
} }
try { try {
const audio = await configureScreenAudioSender( const audio = await peer.waitForOperation(() => configureScreenAudioSender(
this.audioSender, audioSender,
profile.screenAudioQuality, profile.screenAudioQuality,
); ));
if (this.peer !== peer) return false; if (this.peer !== peer) return false;
this.audioWarning = audioSenderParameterWarning(audio); this.audioWarning = audioSenderParameterWarning(audio);
return true; return true;
} catch { } catch {
if (this.peer !== peer) return false;
this.audioWarning = { key: "host.fail.sfuAudioParams" }; this.audioWarning = { key: "host.fail.sfuAudioParams" };
return false; return false;
} }
@@ -527,7 +532,7 @@ export class SfuPublisher {
if (!peer || !video || !sender || this.statsInFlight === stats) return; if (!peer || !video || !sender || this.statsInFlight === stats) return;
this.statsInFlight = stats; this.statsInFlight = stats;
try { try {
const report = await peer.pc.getStats(); const report = await peer.waitForOperation(() => peer.pc.getStats());
debugRtcStats(peer.pc, report); debugRtcStats(peer.pc, report);
debugTrack(video, { event: "sfu-sample" }); debugTrack(video, { event: "sfu-sample" });
if (this.peer !== peer || this.video !== video || this.stats !== stats) if (this.peer !== peer || this.video !== video || this.stats !== stats)
+2 -2
View File
@@ -73,10 +73,10 @@ export class SfuSubscriber {
await peer.acceptSignal(message, async (description) => { await peer.acceptSignal(message, async (description) => {
if (description.type !== "offer") if (description.type !== "offer")
throw new Error("SFU subscriber expected an offer"); throw new Error("SFU subscriber expected an offer");
await peer.pc.setRemoteDescription(description); await peer.waitForOperation(() => peer.pc.setRemoteDescription(description));
if (this.peer !== peer) return; if (this.peer !== peer) return;
await peer.sendDescription( await peer.sendDescription(
preferScreenAudioStereo(await peer.pc.createAnswer()), preferScreenAudioStereo(await peer.waitForOperation(() => peer.pc.createAnswer())),
); );
this.restartPending = false; this.restartPending = false;
}); });
+21
View File
@@ -0,0 +1,21 @@
// Closing a PeerConnection may leave its operations-chain promises pending.
// The connection owner cancels its waits; late results cannot resume its work.
// https://www.w3.org/TR/webrtc/#dfn-chain-an-asynchronous-operation
export async function waitForConnectionOperation<T>(
signal: AbortSignal,
operation: () => Promise<T>,
): Promise<T> {
signal.throwIfAborted();
let abort!: () => void;
const stopped = new Promise<never>((_, reject) => {
abort = () => reject(signal.reason);
signal.addEventListener("abort", abort, { once: true });
});
try {
const result = await Promise.race([operation(), stopped]);
signal.throwIfAborted();
return result;
} finally {
signal.removeEventListener("abort", abort);
}
}
+44 -25
View File
@@ -1,6 +1,7 @@
import type { CopyKey } from "../ui/copy"; import type { CopyKey } from "../ui/copy";
import type { IceConfig, SignalPayload } from "../../shared/protocol"; import type { IceConfig, SignalPayload } from "../../shared/protocol";
import { createOpaqueId } from "../lib/opaque-id"; import { createOpaqueId } from "../lib/opaque-id";
import { waitForConnectionOperation } from "./connection-operation";
import type { BrowserEncodingPool, BrowserPooledSender } from "../media/browser-encoding-pool"; import type { BrowserEncodingPool, BrowserPooledSender } from "../media/browser-encoding-pool";
import { BrowserEncodingOutput, encodedStreams, supportsBrowserEncoding } from "../media/browser-encoding-output"; import { BrowserEncodingOutput, encodedStreams, supportsBrowserEncoding } from "../media/browser-encoding-output";
import { debugError, debugEvent } from "../lib/debug"; import { debugError, debugEvent } from "../lib/debug";
@@ -91,7 +92,8 @@ export class HostPeer {
private appliedAudioSenderParameters: private appliedAudioSenderParameters:
| AudioSenderParameterReadback | AudioSenderParameterReadback
| null = null; | null = null;
private disposed = false; private readonly lifetime = new AbortController();
private get disposed(): boolean { return this.lifetime.signal.aborted; }
private profileRevision = 0; private profileRevision = 0;
private negotiationEpoch = 0; private negotiationEpoch = 0;
private ordinaryAnswerEpoch: number | null = null; private ordinaryAnswerEpoch: number | null = null;
@@ -252,16 +254,18 @@ export class HostPeer {
try { try {
try { try {
if (videoChanged) await videoSender.replaceTrack(nextVideoTrack); if (videoChanged) await this.waitForOperation(() => videoSender.replaceTrack(nextVideoTrack));
await audioSender.replaceTrack(nextAudioTrack); await this.waitForOperation(() => audioSender.replaceTrack(nextAudioTrack));
if (audioDirectionChanged) { if (audioDirectionChanged) {
audioTransceiver.direction = nextAudioDirection; audioTransceiver.direction = nextAudioDirection;
} }
} catch (error) { } catch (error) {
if (this.disposed) return false;
const [videoRollback] = await Promise.allSettled([ const [videoRollback] = await Promise.allSettled([
videoChanged ? videoSender.replaceTrack(previousVideoTrack) : Promise.resolve(), videoChanged ? this.waitForOperation(() => videoSender.replaceTrack(previousVideoTrack)) : Promise.resolve(),
audioSender.replaceTrack(previousAudioTrack), this.waitForOperation(() => audioSender.replaceTrack(previousAudioTrack)),
]); ]);
if (this.disposed) return false;
if (audioDirectionChanged) { if (audioDirectionChanged) {
audioTransceiver.direction = previousAudioDirection; audioTransceiver.direction = previousAudioDirection;
} }
@@ -294,6 +298,7 @@ export class HostPeer {
video: videoChanged && this.connection.connectionState === "connected", video: videoChanged && this.connection.connectionState === "connected",
audio: true, audio: true,
}); });
if (this.disposed) return false;
this.snapshot = { ...this.snapshot, error: null }; this.snapshot = { ...this.snapshot, error: null };
this.emit(); this.emit();
return !audioDirectionChanged || (await this.createOffer(false)); return !audioDirectionChanged || (await this.createOffer(false));
@@ -369,7 +374,12 @@ export class HostPeer {
return false; return false;
} }
if (updateCaptureConstraints) { if (updateCaptureConstraints) {
await applyVideoCaptureProfile(videoTrack, profile); try {
await this.waitForOperation(() => applyVideoCaptureProfile(videoTrack, profile));
} catch (error) {
if (this.disposed) return false;
throw error;
}
if ( if (
this.disposed || this.disposed ||
this.senderVideoTrack !== videoTrack || this.senderVideoTrack !== videoTrack ||
@@ -434,7 +444,7 @@ export class HostPeer {
this.pendingCandidates.push(payload.candidate); this.pendingCandidates.push(payload.candidate);
} }
} else { } else {
await addRemoteIceCandidate(this.connection, payload.candidate); await this.waitForOperation(() => addRemoteIceCandidate(this.connection, payload.candidate));
} }
}); });
} }
@@ -490,7 +500,7 @@ export class HostPeer {
if (this.disposed) { if (this.disposed) {
return; return;
} }
this.disposed = true; this.lifetime.abort();
this.nextNegotiationEpoch(); this.nextNegotiationEpoch();
this.ordinaryAnswerEpoch = null; this.ordinaryAnswerEpoch = null;
if (this.statsTimer !== null) { if (this.statsTimer !== null) {
@@ -552,7 +562,7 @@ export class HostPeer {
if (!owns()) return; if (!owns()) return;
const request = sender.setParameters as (parameters: RTCRtpSendParameters, const request = sender.setParameters as (parameters: RTCRtpSendParameters,
options: { encodingOptions: Array<{ keyFrame: boolean }> }) => Promise<void>; options: { encodingOptions: Array<{ keyFrame: boolean }> }) => Promise<void>;
await request.call(sender, sender.getParameters(), { encodingOptions: [{ keyFrame: true }] }); await this.waitForOperation(() => request.call(sender, sender.getParameters(), { encodingOptions: [{ keyFrame: true }] }));
}); });
binding = this.videoPool.create(source, sender, this.connection, this.desiredProfile, this.encodedOutput, binding = this.videoPool.create(source, sender, this.connection, this.desiredProfile, this.encodedOutput,
() => this.enqueueSenderMutation(async () => { () => this.enqueueSenderMutation(async () => {
@@ -561,12 +571,12 @@ export class HostPeer {
const next = binding?.carrierScale() === undefined ? cloneSenderVideoTrack(source) : this.encodedOutput!.track; const next = binding?.carrierScale() === undefined ? cloneSenderVideoTrack(source) : this.encodedOutput!.track;
try { try {
this.applyPausedState(next, null); this.applyPausedState(next, null);
await applyVideoCaptureProfile(next, this.desiredProfile); await this.waitForOperation(() => applyVideoCaptureProfile(next, this.desiredProfile));
if (!owns()) { if (next !== this.encodedOutput?.track) next.stop(); return false; } if (!owns()) { if (next !== this.encodedOutput?.track) next.stop(); return false; }
if (next !== previous) await sender.replaceTrack(next); if (next !== previous) await this.waitForOperation(() => sender.replaceTrack(next));
await configureVideoSender(sender, await this.waitForOperation(() => configureVideoSender(sender,
this.startupVideoProfilePending ? startupVideoProfile(this.desiredProfile) : this.desiredProfile, this.startupVideoProfilePending ? startupVideoProfile(this.desiredProfile) : this.desiredProfile,
binding?.carrierScale()); binding?.carrierScale()));
if (!owns()) throw new DOMException("Retired Browser pool binding", "AbortError"); if (!owns()) throw new DOMException("Retired Browser pool binding", "AbortError");
this.senderVideoTrack = next; this.senderVideoTrack = next;
if (previous !== next && previous !== this.encodedOutput?.track) previous.stop(); if (previous !== next && previous !== this.encodedOutput?.track) previous.stop();
@@ -574,9 +584,10 @@ export class HostPeer {
return true; return true;
} catch (error) { } catch (error) {
if (!this.disposed && sender.track !== previous) { if (!this.disposed && sender.track !== previous) {
try { await sender.replaceTrack(previous); } catch { this.dispose(); } try { await this.waitForOperation(() => sender.replaceTrack(previous)); } catch { this.dispose(); }
} }
if (next !== previous && next !== this.encodedOutput?.track) next.stop(); if (next !== previous && next !== this.encodedOutput?.track) next.stop();
if (this.disposed) return false;
debugError("encoding-pool", "carrier-attachment-failed", error, { connectionId: this.connectionId }); debugError("encoding-pool", "carrier-attachment-failed", error, { connectionId: this.connectionId });
return false; return false;
} }
@@ -596,6 +607,10 @@ export class HostPeer {
}); });
} }
private waitForOperation<T>(operation: () => Promise<T>): Promise<T> {
return waitForConnectionOperation(this.lifetime.signal, operation);
}
private enqueueSenderMutation<T>(operation: () => Promise<T>): Promise<T> { private enqueueSenderMutation<T>(operation: () => Promise<T>): Promise<T> {
const result = this.senderMutationTail.then(operation); const result = this.senderMutationTail.then(operation);
this.senderMutationTail = result.then( this.senderMutationTail = result.then(
@@ -634,11 +649,11 @@ export class HostPeer {
if (restart) { if (restart) {
this.connection.restartIce(); this.connection.restartIce();
} }
const offer = await this.connection.createOffer(); const offer = await this.waitForOperation(() => this.connection.createOffer());
if (!this.ownsLocalOffer(epoch)) { if (!this.ownsLocalOffer(epoch)) {
return false; return false;
} }
await this.connection.setLocalDescription(offer); await this.waitForOperation(() => this.connection.setLocalDescription(offer));
if ( if (
!this.ownsLocalOffer(epoch) || !this.ownsLocalOffer(epoch) ||
!this.connection.localDescription !this.connection.localDescription
@@ -676,7 +691,7 @@ export class HostPeer {
private async flushCandidates(): Promise<void> { private async flushCandidates(): Promise<void> {
const candidates = this.pendingCandidates.splice(0); const candidates = this.pendingCandidates.splice(0);
for (const candidate of candidates) { for (const candidate of candidates) {
await addRemoteIceCandidate(this.connection, candidate); await this.waitForOperation(() => addRemoteIceCandidate(this.connection, candidate));
} }
} }
@@ -688,7 +703,7 @@ export class HostPeer {
return; return;
} }
try { try {
await this.connection.setRemoteDescription(payload.description); await this.waitForOperation(() => this.connection.setRemoteDescription(payload.description));
if (!this.ownsAnswer(epoch)) { if (!this.ownsAnswer(epoch)) {
return; return;
} }
@@ -744,7 +759,7 @@ export class HostPeer {
this.statsInFlight = true; this.statsInFlight = true;
const statsAccumulator = this.statsAccumulator; const statsAccumulator = this.statsAccumulator;
try { try {
const metricsPromise = collectConnectionMetrics( const metricsPromise = this.waitForOperation(() => collectConnectionMetrics(
this.connection, this.connection,
"send", "send",
statsAccumulator, statsAccumulator,
@@ -752,7 +767,7 @@ export class HostPeer {
trackIdentifier: captureTrack.id, trackIdentifier: captureTrack.id,
audioTrackIdentifier: captureAudioTrack?.id ?? null, audioTrackIdentifier: captureAudioTrack?.id ?? null,
}, },
); ));
const capture = captureMetrics(captureTrack); const capture = captureMetrics(captureTrack);
debugTrack(captureTrack, { connectionId: this.connectionId, event: "sample" }); debugTrack(captureTrack, { connectionId: this.connectionId, event: "sample" });
const metrics = { ...(await metricsPromise), ...capture }; const metrics = { ...(await metricsPromise), ...capture };
@@ -792,6 +807,7 @@ export class HostPeer {
} }
private setError(error: unknown, key: CopyKey): void { private setError(error: unknown, key: CopyKey): void {
if (this.disposed) return;
debugError("webrtc", "sender-failed", error, { connectionId: this.connectionId, reason: key }); debugError("webrtc", "sender-failed", error, { connectionId: this.connectionId, reason: key });
this.snapshot = { ...this.snapshot, error: { key } }; this.snapshot = { ...this.snapshot, error: { key } };
this.emit(); this.emit();
@@ -820,8 +836,9 @@ export class HostPeer {
if (mutation.video) { if (mutation.video) {
try { try {
senderParameters = await configureVideoSender(sender, profile, this.pooledVideo?.carrierScale()); senderParameters = await this.waitForOperation(() => configureVideoSender(sender, profile, this.pooledVideo?.carrierScale()));
} catch (error) { } catch (error) {
if (this.disposed) return false;
videoSucceeded = false; videoSucceeded = false;
debugError("webrtc", "sender-parameters-failed", error, { connectionId: this.connectionId, profileRevision, requested: profile }); debugError("webrtc", "sender-parameters-failed", error, { connectionId: this.connectionId, profileRevision, requested: profile });
} }
@@ -835,19 +852,21 @@ export class HostPeer {
return false; return false;
} }
if (mutation.video && videoSucceeded && this.pooledVideo) { const pooledVideo = this.pooledVideo;
try { await this.pooledVideo.updateProfile(this.desiredProfile); } if (mutation.video && videoSucceeded && pooledVideo) {
try { await this.waitForOperation(() => pooledVideo.updateProfile(this.desiredProfile)); }
catch { videoSucceeded = false; } catch { videoSucceeded = false; }
} }
if (mutation.audio && audioSender) { if (mutation.audio && audioSender) {
if (audioSender.track) { if (audioSender.track) {
try { try {
audioSenderParameters = await configureScreenAudioSender( audioSenderParameters = await this.waitForOperation(() => configureScreenAudioSender(
audioSender, audioSender,
profile.screenAudioQuality, profile.screenAudioQuality,
); ));
} catch (error) { } catch (error) {
if (this.disposed) return false;
audioSucceeded = false; audioSucceeded = false;
debugError("webrtc", "audio-parameters-failed", error, { connectionId: this.connectionId, profileRevision }); debugError("webrtc", "audio-parameters-failed", error, { connectionId: this.connectionId, profileRevision });
} }
+77 -77
View File
@@ -268,114 +268,111 @@ function hasProbeWindow(
return current.timestamp - baseline.timestamp >= PREFLIGHT_WARMUP_MS; return current.timestamp - baseline.timestamp >= PREFLIGHT_WARMUP_MS;
} }
function delay(milliseconds: number, signal?: AbortSignal): Promise<void> {
return new Promise((resolve, reject) => {
if (signal?.aborted) {
reject(signal.reason);
return;
}
const onAbort = () => {
window.clearTimeout(timer);
reject(signal?.reason);
};
const timer = window.setTimeout(() => {
signal?.removeEventListener("abort", onAbort);
resolve();
}, milliseconds);
signal?.addEventListener("abort", onAbort, { once: true });
});
}
async function runH264Probe( async function runH264Probe(
track: MediaStreamTrack, track: MediaStreamTrack,
profile: QualityProfile, profile: QualityProfile,
signal?: AbortSignal, signal?: AbortSignal,
): Promise<boolean> { ): Promise<boolean> {
const senderConnection = new RTCPeerConnection({ iceServers: [] }); const senderConnection = new RTCPeerConnection({ iceServers: [] });
const receiverConnection = new RTCPeerConnection({ iceServers: [] }); let receiver: RTCPeerConnection | undefined;
// This in-process probe can start encoding as soon as one local pair works; let pollTimer: number | undefined;
// gathering every candidate must not extend the codec decision. let stopReason: Error | undefined;
const senderCandidates: RTCIceCandidate[] = []; let stop!: (reason: Error) => void;
const receiverCandidates: RTCIceCandidate[] = []; const stopped = new Promise<never>((_, reject) => {
let senderRemoteReady = false; stop = (reason) => {
let receiverRemoteReady = false; stopReason ??= reason;
let iceFailed = false; reject(stopReason);
const addCandidate = async ( };
target: RTCPeerConnection,
candidate: RTCIceCandidate,
): Promise<void> => {
try {
await addRemoteIceCandidate(target, candidate);
} catch {
iceFailed = true;
}
};
senderConnection.addEventListener("icecandidate", (event) => {
if (!event.candidate) return;
if (receiverRemoteReady) {
void addCandidate(receiverConnection, event.candidate);
} else {
senderCandidates.push(event.candidate);
}
}); });
receiverConnection.addEventListener("icecandidate", (event) => { // Closing a PeerConnection need not settle its pending SDP operations.
if (!event.candidate) return; // Every await belongs to this one budget, including setup and statistics.
if (senderRemoteReady) { const wait = async <T>(operation: () => Promise<T>): Promise<T> => {
void addCandidate(senderConnection, event.candidate); if (stopReason) throw stopReason;
} else { const value = await Promise.race([operation(), stopped]);
receiverCandidates.push(event.candidate); if (stopReason) throw stopReason;
} return value;
});
const abort = () => {
senderConnection.close();
receiverConnection.close();
}; };
const abort = () => stop(new DOMException("Codec probe cancelled", "AbortError"));
const timer = window.setTimeout(
() => stop(new Error("Codec probe timed out")), PREFLIGHT_DEADLINE_MS,
);
signal?.addEventListener("abort", abort, { once: true }); signal?.addEventListener("abort", abort, { once: true });
try { try {
const receiverConnection = receiver = new RTCPeerConnection({ iceServers: [] });
// This in-process probe can start encoding as soon as one local pair works;
// gathering every candidate must not extend the codec decision.
const senderCandidates: RTCIceCandidate[] = [];
const receiverCandidates: RTCIceCandidate[] = [];
let senderRemoteReady = false;
let receiverRemoteReady = false;
let iceFailed = false;
const addCandidate = async (
target: RTCPeerConnection,
candidate: RTCIceCandidate,
): Promise<void> => {
try {
await addRemoteIceCandidate(target, candidate);
} catch {
iceFailed = true;
}
};
senderConnection.addEventListener("icecandidate", (event) => {
if (!event.candidate) return;
if (receiverRemoteReady) {
void addCandidate(receiverConnection, event.candidate);
} else {
senderCandidates.push(event.candidate);
}
});
receiverConnection.addEventListener("icecandidate", (event) => {
if (!event.candidate) return;
if (senderRemoteReady) {
void addCandidate(senderConnection, event.candidate);
} else {
receiverCandidates.push(event.candidate);
}
});
const transceiver = senderConnection.addTransceiver(track, { const transceiver = senderConnection.addTransceiver(track, {
direction: "sendonly", direction: "sendonly",
}); });
if (!applyH264ProbeCodec(transceiver)) { if (!applyH264ProbeCodec(transceiver)) {
return false; return false;
} }
await configureVideoSender( await wait(() => configureVideoSender(
transceiver.sender, transceiver.sender,
startupVideoProfile(profile), startupVideoProfile(profile),
); ));
const deadline = Date.now() + PREFLIGHT_DEADLINE_MS; const offer = await wait(() => senderConnection.createOffer());
await senderConnection.setLocalDescription( await wait(() => senderConnection.setLocalDescription(offer));
await senderConnection.createOffer(), await wait(() => receiverConnection.setRemoteDescription(
);
await receiverConnection.setRemoteDescription(
senderConnection.localDescription!, senderConnection.localDescription!,
); ));
receiverRemoteReady = true; receiverRemoteReady = true;
await Promise.all( await wait(() => Promise.all(
senderCandidates.splice(0).map((candidate) => senderCandidates.splice(0).map((candidate) =>
addCandidate(receiverConnection, candidate), addCandidate(receiverConnection, candidate),
), ),
); ));
await receiverConnection.setLocalDescription( const answer = await wait(() => receiverConnection.createAnswer());
await receiverConnection.createAnswer(), await wait(() => receiverConnection.setLocalDescription(answer));
); await wait(() => senderConnection.setRemoteDescription(
await senderConnection.setRemoteDescription(
receiverConnection.localDescription!, receiverConnection.localDescription!,
); ));
senderRemoteReady = true; senderRemoteReady = true;
await Promise.all( await wait(() => Promise.all(
receiverCandidates.splice(0).map((candidate) => receiverCandidates.splice(0).map((candidate) =>
addCandidate(senderConnection, candidate), addCandidate(senderConnection, candidate),
), ),
); ));
let warmupBaseline: H264ProbeSample | null = null; let warmupBaseline: H264ProbeSample | null = null;
let measurementBaseline: H264ProbeSample | null = null; let measurementBaseline: H264ProbeSample | null = null;
while (Date.now() < deadline) { for (;;) {
if (signal?.aborted || track.readyState === "ended" || iceFailed) { if (signal?.aborted || track.readyState === "ended" || iceFailed) {
return false; return false;
} }
const sample = readH264ProbeSample( const sample = readH264ProbeSample(
await transceiver.sender.getStats(), await wait(() => transceiver.sender.getStats()),
); );
if (sample) { if (sample) {
warmupBaseline ??= sample; warmupBaseline ??= sample;
@@ -396,13 +393,16 @@ async function runH264Probe(
} }
} }
} }
await delay(PREFLIGHT_POLL_MS, signal); await wait(() => new Promise<void>((resolve) => {
pollTimer = window.setTimeout(resolve, PREFLIGHT_POLL_MS);
}));
} }
return false;
} finally { } finally {
window.clearTimeout(timer);
window.clearTimeout(pollTimer);
signal?.removeEventListener("abort", abort); signal?.removeEventListener("abort", abort);
senderConnection.close(); senderConnection.close();
receiverConnection.close(); receiver?.close();
} }
} }
@@ -411,7 +411,7 @@ export async function preferredVideoCodecForTrack(
profile: QualityProfile, profile: QualityProfile,
signal?: AbortSignal, signal?: AbortSignal,
): Promise<BrowserVideoCodec> { ): Promise<BrowserVideoCodec> {
if (track.kind !== "video" || track.readyState === "ended") { if (signal?.aborted || track.kind !== "video" || track.readyState === "ended") {
return "vp8"; return "vp8";
} }
let probe: H264ProbeTrack | null = null; let probe: H264ProbeTrack | null = null;
+146
View File
@@ -0,0 +1,146 @@
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { BrowserEncodingPool } from "../src/client/media/browser-encoding-pool";
import { BrowserEncodingProducer } from "../src/client/media/browser-encoding-producer";
import type { BrowserEncodingOutput } from "../src/client/media/browser-encoding-output";
import { QUALITY_PROFILES } from "../src/client/media/quality";
const codec = { mimeType: "video/VP8", clockRate: 90_000 };
const profile = QUALITY_PROFILES["1080p30"];
const source = { kind: "video", readyState: "live", enabled: true, id: "source" } as MediaStreamTrack;
const producers = new Map<BrowserEncodingProducer, { budget: number; bitrate: number; bytes: number; frames: number }>();
let pool: BrowserEncodingPool;
function stats(row: Record<string, unknown>): RTCStatsReport {
return new Map([
["video", { id: "video", timestamp: Date.now(), type: "outbound-rtp", kind: "video", codecId: "codec", ...row }],
["codec", { id: "codec", timestamp: Date.now(), type: "codec", ...codec }],
]) as unknown as RTCStatsReport;
}
function member(budget = 2_000_000) {
const demand = { budget };
let producerId: string | null = null;
const select = vi.fn((id: string, _requestKey: () => Promise<void>, selected: () => void) => {
producerId = id;
selected();
});
const encoded = {
select, push: vi.fn(), passthrough: vi.fn(), setPaused: vi.fn(),
snapshot: () => ({ frames: producerId ? 30 : 0, width: 1920, height: 1080, timestamp: Date.now(), lastProducerId: producerId }),
} as unknown as BrowserEncodingOutput;
const connection = { connectionState: "connected", getStats: async () => stats({ targetBitrate: demand.budget }) };
const fatal = vi.fn();
const handle = pool.create(source, {} as RTCRtpSender, connection as RTCPeerConnection, profile,
encoded, async () => true, async () => undefined, fatal)!;
return { demand, select, fatal, handle, producerId: () => producerId };
}
async function sample(count = 1) {
await vi.advanceTimersByTimeAsync(500 * count);
}
beforeEach(() => {
vi.useFakeTimers();
vi.stubGlobal("RTCRtpSender", { getCapabilities: () => ({ codecs: [codec] }) });
producers.clear();
vi.spyOn(BrowserEncodingProducer.prototype, "start").mockImplementation(async function (this: BrowserEncodingProducer, budget) {
producers.set(this, { budget: budget!, bitrate: budget! / 2, bytes: 0, frames: 0 });
});
vi.spyOn(BrowserEncodingProducer.prototype, "report").mockImplementation(async function (this: BrowserEncodingProducer) {
const output = producers.get(this)!;
output.bytes += output.bitrate / 16;
output.frames += 15;
return stats({ bytesSent: output.bytes, framesEncoded: output.frames, frameWidth: 1920, frameHeight: 1080,
qualityLimitationReason: "none" });
});
vi.spyOn(BrowserEncodingProducer.prototype, "update").mockImplementation(async function (this: BrowserEncodingProducer, _profile, budget) {
producers.get(this)!.budget = budget!;
});
vi.spyOn(BrowserEncodingProducer.prototype, "setPaused").mockImplementation(() => undefined);
vi.spyOn(BrowserEncodingProducer.prototype, "dispose").mockImplementation(() => undefined);
pool = new BrowserEncodingPool();
});
afterEach(() => {
pool.dispose();
vi.restoreAllMocks();
vi.unstubAllGlobals();
vi.useRealTimers();
});
describe("Browser encoding pool ownership", () => {
it("keeps a shared encoder through a bitrate spike with unchanged native demand", async () => {
const first = member(), second = member();
await sample(4);
expect(producers.size).toBe(1);
const [producer, output] = [...producers][0];
expect(first.producerId()).toBe(producer.id);
expect(second.producerId()).toBe(producer.id);
output.bitrate = output.budget + 16;
await sample();
output.bitrate = output.budget / 2;
await sample(2);
expect(producers.size).toBe(1);
expect(first.select).toHaveBeenCalledTimes(1);
expect(second.select).toHaveBeenCalledTimes(1);
expect(producer.dispose).not.toHaveBeenCalled();
});
it("updates shared demand in place when all children need a lower rate", async () => {
const first = member(), second = member();
await sample(4);
const [producer, output] = [...producers][0];
first.demand.budget = second.demand.budget = 500_000;
await sample(2);
expect(producers.size).toBe(1);
expect(output.budget).toBe(500_000);
expect(producer.update).toHaveBeenCalledWith(profile, 500_000);
expect(first.select).toHaveBeenCalledTimes(1);
expect(second.select).toHaveBeenCalledTimes(1);
});
it("gives a weaker child its own rate owner and reunites it after recovery", async () => {
const strong = member(), weak = member();
await sample(4);
const [original, output] = [...producers][0];
weak.demand.budget = 500_000;
await sample(3);
expect(producers.size).toBe(2);
expect(strong.producerId()).toBe(original.id);
expect(weak.producerId()).not.toBe(original.id);
expect(output.budget).toBe(2_000_000);
expect(strong.select).toHaveBeenCalledTimes(1);
weak.demand.budget = 2_000_000;
await sample(3);
expect(weak.producerId()).toBe(original.id);
expect(producers.size).toBe(2);
expect(strong.fatal).not.toHaveBeenCalled();
expect(weak.fatal).not.toHaveBeenCalled();
});
it("starts protection at committed output, ignoring retired selection callbacks", async () => {
const begin = vi.spyOn(BrowserEncodingProducer.prototype, "beginOutput");
const first = member();
let selected: (() => void) | undefined;
first.select.mockImplementation((_id, _requestKey, commit) => { selected = commit; });
await sample(3);
expect(selected).toBeDefined();
expect(begin).not.toHaveBeenCalled();
selected!();
expect(begin).toHaveBeenCalledOnce();
const late = member();
let retired: (() => void) | undefined;
late.select.mockImplementation((_id, _requestKey, commit) => { retired = commit; });
await sample(2);
expect(retired).toBeDefined();
late.handle.dispose();
retired!();
expect(begin).toHaveBeenCalledOnce();
});
});
+68 -2
View File
@@ -1,6 +1,6 @@
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { BrowserEncodingProducer } from "../src/client/media/browser-encoding-producer"; import { BrowserEncodingProducer } from "../src/client/media/browser-encoding-producer";
import { QUALITY_PROFILES } from "../src/client/media/quality"; import { configureVideoSender, QUALITY_PROFILES } from "../src/client/media/quality";
vi.mock("../src/client/media/quality", async (original) => ({ vi.mock("../src/client/media/quality", async (original) => ({
...await original<typeof import("../src/client/media/quality")>(), ...await original<typeof import("../src/client/media/quality")>(),
@@ -9,6 +9,7 @@ vi.mock("../src/client/media/quality", async (original) => ({
})); }));
class Track extends EventTarget { class Track extends EventTarget {
readonly id = "capture";
enabled = true; enabled = true;
readyState = "live"; readyState = "live";
contentHint = "motion"; contentHint = "motion";
@@ -25,9 +26,16 @@ class Connection {
localDescription: RTCSessionDescriptionInit | null = null; localDescription: RTCSessionDescriptionInit | null = null;
remoteDescription: RTCSessionDescriptionInit | null = null; remoteDescription: RTCSessionDescriptionInit | null = null;
readonly close = vi.fn(() => { this.connectionState = "closed"; }); readonly close = vi.fn(() => { this.connectionState = "closed"; });
private frames!: ReadableStreamDefaultController<RTCEncodedVideoFrame>;
readonly getStats = vi.fn(async () => new Map<string, object>([
["video", { type: "outbound-rtp", kind: "video", mediaSourceId: "source", framesEncoded: 100 }],
["source", { type: "media-source", trackIdentifier: "capture" }],
]) as unknown as RTCStatsReport);
readonly sender = { readonly sender = {
getParameters: () => ({ encodings: [{}] }),
setParameters: vi.fn(async () => undefined),
createEncodedStreams: () => ({ createEncodedStreams: () => ({
readable: new ReadableStream<RTCEncodedVideoFrame>(), readable: new ReadableStream<RTCEncodedVideoFrame>({ start: controller => { this.frames = controller; } }),
writable: new WritableStream<RTCEncodedVideoFrame>(), writable: new WritableStream<RTCEncodedVideoFrame>(),
}), }),
}; };
@@ -38,6 +46,10 @@ class Connection {
async setLocalDescription(value: RTCSessionDescriptionInit) { this.localDescription = value; } async setLocalDescription(value: RTCSessionDescriptionInit) { this.localDescription = value; }
async setRemoteDescription(value: RTCSessionDescriptionInit) { this.remoteDescription = value; } async setRemoteDescription(value: RTCSessionDescriptionInit) { this.remoteDescription = value; }
state(value: string) { this.connectionState = value; this.onconnectionstatechange?.(); } state(value: string) { this.connectionState = value; this.onconnectionstatechange?.(); }
async frame(bytes = 1) {
this.frames.enqueue({ data: new ArrayBuffer(bytes) } as RTCEncodedVideoFrame);
await vi.advanceTimersByTimeAsync(0);
}
} }
const producers: BrowserEncodingProducer[] = []; const producers: BrowserEncodingProducer[] = [];
@@ -51,6 +63,7 @@ function createProducer() {
} }
beforeEach(() => { beforeEach(() => {
vi.clearAllMocks();
vi.useFakeTimers(); vi.useFakeTimers();
Connection.instances = []; Connection.instances = [];
vi.stubGlobal("RTCPeerConnection", Connection); vi.stubGlobal("RTCPeerConnection", Connection);
@@ -66,6 +79,37 @@ afterEach(async () => {
}); });
describe("Browser encoding producer startup", () => { describe("Browser encoding producer startup", () => {
it("protects first publication instead of spending the startup frames on local warmup", async () => {
const { producer } = createProducer();
await producer.start();
for (const connection of Connection.instances) connection.state("connected");
const send = Connection.instances[0];
const preference = () => vi.mocked(configureVideoSender).mock.lastCall![1].degradationPreference;
const report = async () => { await producer.report(); await vi.advanceTimersByTimeAsync(0); };
for (let i = 0; i < 10; i++) await send.frame();
await report();
expect(preference()).toBe("maintain-resolution");
producer.beginOutput();
producer.setPaused(true);
for (let i = 0; i < 5; i++) await send.frame();
producer.setPaused(false);
await send.frame(0);
for (let i = 0; i < 4; i++) await send.frame();
await report();
expect(preference()).toBe("maintain-resolution");
// Another child's attachment must not restart this producer's protection.
producer.beginOutput();
await send.frame();
await report();
expect(preference()).toBe("balanced");
producer.beginOutput();
await report();
expect(preference()).toBe("balanced");
});
it.each(["queued", "active"])("keeps the producer after a %s candidate rejection", async (phase) => { it.each(["queued", "active"])("keeps the producer after a %s candidate rejection", async (phase) => {
const { producer, failed } = createProducer(); const { producer, failed } = createProducer();
const starting = producer.start(); const starting = producer.start();
@@ -123,4 +167,26 @@ describe("Browser encoding producer startup", () => {
expect(next.failed).not.toHaveBeenCalled(); expect(next.failed).not.toHaveBeenCalled();
expect(next.producer.track?.readyState).toBe("live"); expect(next.producer.track?.readyState).toBe("live");
}); });
it("routes keyframe request failure through the producer owner, not the outgoing connection", async () => {
const { producer, source, failed } = createProducer();
await producer.start();
for (const connection of Connection.instances) connection.state("connected");
Connection.instances[0].sender.setParameters.mockRejectedValueOnce(new Error("Encoder failed"));
await expect(producer.requestKey()).resolves.toBeUndefined();
expect(failed).toHaveBeenCalledOnce();
expect(producer.track).toBeNull();
expect(source.stop).not.toHaveBeenCalled();
});
it("ignores a keyframe request during ordinary recovery after the producer has retired", async () => {
const { producer, failed } = createProducer();
await producer.start();
Connection.instances[0].state("failed");
expect(failed).toHaveBeenCalledOnce();
await expect(producer.requestKey()).resolves.toBeUndefined();
expect(failed).toHaveBeenCalledOnce();
});
}); });
+29
View File
@@ -14,6 +14,7 @@ import {
getSiteAccess, getSiteAccess,
getOptionalRuntimeCapabilities, getOptionalRuntimeCapabilities,
replaceOwnedRoom, replaceOwnedRoom,
updateRoomAccess,
} from "../src/client/lib/api.ts"; } from "../src/client/lib/api.ts";
import { import {
defaultHostDisplayName, defaultHostDisplayName,
@@ -739,6 +740,34 @@ describe("client session identity", () => {
}); });
describe("site access API", () => { describe("site access API", () => {
it.each(["zh", "en"] as const)("explains rejected origins across write APIs in %s", async (lang) => {
setCopy({ lang, vis: false });
vi.stubGlobal("fetch", vi.fn<typeof fetch>().mockImplementation(async () =>
new Response(JSON.stringify({ error: "Origin not allowed" }), { status: 403 }),
));
const message = lang === "zh"
? "服务器未允许当前网址 (403)。请联系站点管理员检查 PUBLIC_BASE_URL 和 ALLOWED_ORIGINS。"
: "This address is not allowed by the server (403). Ask the site administrator to check PUBLIC_BASE_URL and ALLOWED_ORIGINS.";
for (const request of [
() => authenticateSiteAccess("password"),
() => createRoom("open", null),
() => replaceOwnedRoom("4321", "host-token", "open", null),
() => updateRoomAccess("4321", "host-token", { action: "rotate-viewer-grant" }),
]) {
await expect(request()).rejects.toMatchObject({ status: 403, message });
}
});
it("does not diagnose arbitrary proxy rejection bodies as an origin mismatch", async () => {
for (const body of [JSON.stringify({ error: "Forbidden" }), "<html>Forbidden</html>"]) {
vi.stubGlobal("fetch", vi.fn<typeof fetch>().mockResolvedValue(new Response(body, { status: 403 })));
await expect(createRoom("open", null)).rejects.toMatchObject({
status: 403,
message: expect.not.stringContaining("ALLOWED_ORIGINS"),
});
}
});
it("checks site access and sends the exact password in a Unicode-capable JSON body", async () => { it("checks site access and sends the exact password in a Unicode-capable JSON body", async () => {
const fetchMock = vi const fetchMock = vi
.fn<typeof fetch>() .fn<typeof fetch>()
+35
View File
@@ -862,6 +862,41 @@ describe("HostPeer source replacement", () => {
expect(nextSenderVideo?.stop).toHaveBeenCalledOnce(); expect(nextSenderVideo?.stop).toHaveBeenCalledOnce();
}); });
it.each(["video", "audio", "rollback"])("retires a replacement blocked in %s without waiting for the browser", async (stage) => {
const video = createTrack("video", "old-video");
const audio = createTrack("audio", "old-audio");
const peer = createPeer(createStream(video, audio));
await peer.start();
const connection = FakePeerConnection.latest!;
const videoSender = connection.senders[0]!;
const audioSender = connection.senders[1]!;
const previousClone = videoSender.track!;
videoSender.deferReplaceCall = stage === "video" ? 1 : stage === "rollback" ? 2 : null;
audioSender.deferReplaceCall = stage === "audio" ? 1 : null;
audioSender.failNextReplace = stage === "rollback";
const nextVideo = createTrack("video", "next-video");
const nextAudio = createTrack("audio", "next-audio");
const replacing = peer.replaceStream(createStream(nextVideo, nextAudio));
await vi.waitFor(() => expect(
stage === "audio" ? audioSender.replaceTrack : videoSender.replaceTrack,
).toHaveBeenCalledTimes(stage === "rollback" ? 2 : 1));
const replacementClone = vi.mocked(nextVideo.clone).mock.results[0]!.value;
const queuedProfile = peer.updateProfile(QUALITY_PROFILES["1080p30"]);
peer.dispose();
await expect(replacing).resolves.toBe(false);
await expect(queuedProfile).resolves.toBe(false);
expect(previousClone.stop).toHaveBeenCalledOnce();
expect(replacementClone.stop).toHaveBeenCalledOnce();
for (const source of [video, audio, nextVideo, nextAudio]) expect(source.stop).not.toHaveBeenCalled();
const audioCalls = audioSender.replaceTrack.mock.calls.length;
const offers = connection.createOfferCallCount;
videoSender.releaseDeferredReplaceTrack();
audioSender.releaseDeferredReplaceTrack();
await new Promise<void>((resolve) => setImmediate(resolve));
expect(audioSender.replaceTrack).toHaveBeenCalledTimes(audioCalls);
expect(connection.createOfferCallCount).toBe(offers);
});
it.each([false, true])("keeps the video sender and quality when only audio changes (failure=%s)", async (fail) => { it.each([false, true])("keeps the video sender and quality when only audio changes (failure=%s)", async (fail) => {
const video = createTrack("video", "same-video"); const video = createTrack("video", "same-video");
const peer = createPeer(createStream(video, null)); const peer = createPeer(createStream(video, null));
+38
View File
@@ -721,6 +721,44 @@ describe("Host quality ownership", () => {
expect(replacement.close).toHaveBeenCalledOnce(); expect(replacement.close).toHaveBeenCalledOnce();
}); });
it.each(["probe", "ingress"])("does not commit Browser codec state after sharing ends during %s", async (stage) => {
const current = fixture(false);
current.activeGenerationRef.current = null;
current.nativeShareGenerationRef.current = null;
const probe = deferred<{ primary: string }>();
const ingress = deferred<void>();
const captured = { getTracks: () => [current.track] };
const resolveStreamVideoCodec = vi.fn(() => probe.promise);
const startBrowserNativeIngress = vi.fn(() => ingress.promise);
const setResolvedVideoCodec = vi.fn();
Object.assign(current.context, {
phase: "idle", captureBrowserSource: async () => captured,
HostAudio: class {}, setStream: vi.fn(), watchCaptureEnd: vi.fn(),
resolveStreamVideoCodec, startBrowserNativeIngress, setResolvedVideoCodec,
});
current.disposeResources.mockImplementation(() => {
current.videoCodecRef.current = { primary: "vp8" };
setResolvedVideoCodec(null);
});
const starting = current.startBrowser();
await vi.waitFor(() => expect(resolveStreamVideoCodec).toHaveBeenCalledOnce());
if (stage === "ingress") {
probe.resolve({ primary: "h264" });
await vi.waitFor(() => expect(startBrowserNativeIngress).toHaveBeenCalledOnce());
}
current.context.endSharing({ key: "host.notice.stopped" });
setResolvedVideoCodec.mockClear();
probe.resolve({ primary: "h264" });
ingress.resolve();
await starting;
expect(current.videoCodecRef.current.primary).toBe("vp8");
expect(setResolvedVideoCodec).not.toHaveBeenCalled();
expect(startBrowserNativeIngress).toHaveBeenCalledTimes(stage === "ingress" ? 1 : 0);
expect(current.track.stop).toHaveBeenCalled();
expect(current.createRoom).not.toHaveBeenCalled();
expect(current.roomMutationRef.current).toBeNull();
});
it("retires a cancelled start ACK before publishing audio, codec or bridge state", async () => { it("retires a cancelled start ACK before publishing audio, codec or bridge state", async () => {
const current = fixture(); const current = fixture();
current.nativeShareGenerationRef.current = null; current.nativeShareGenerationRef.current = null;
+28
View File
@@ -906,6 +906,34 @@ describe("minimal route transition contracts", () => {
expect(publishers[1]?.disconnect).toHaveBeenCalled(); expect(publishers[1]?.disconnect).toHaveBeenCalled();
}); });
it.each(["disconnect", "resync"])("cancels pending Host preparation before joining its queue during %s", async (reason) => {
let cancel!: (result: boolean) => void;
const activating = new Promise<boolean>(resolve => { cancel = resolve; });
const publisher = createFakePublisher([], "pending");
publisher.activate.mockImplementation(() => activating);
publisher.disconnect.mockImplementation(async () => { cancel(false); });
const route = new HostSfuRoute({
getStream: () => ({}) as MediaStream,
getProfile: () => QUALITY_PROFILES["720p30"],
getVideoCodec: () => "vp8",
reconcileChildren: () => undefined,
send: () => true,
createPublisher: () => publisher,
});
await route.acceptAndWait({
revision: 2, phase: "active", assignment: hostAssignment("pending"),
});
const preparing = route.acceptConfig(sfuConfig(2, "pending"));
await vi.waitFor(() => expect(publisher.activate).toHaveBeenCalledOnce());
const retiring = reason === "disconnect" ? route.disconnect() : route.resyncAuthoritative({
revision: 1, phase: "active", assignment: hostAssignment(null),
});
expect(publisher.disconnect).toHaveBeenCalled();
await Promise.all([preparing, retiring]);
expect(publisher.setPaused).not.toHaveBeenCalled();
await route.disconnect();
});
it("ignores a retired publisher failure after same-generation resync", async () => { it("ignores a retired publisher failure after same-generation resync", async () => {
let releaseOldConnect!: () => void; let releaseOldConnect!: () => void;
const oldConnectGate = new Promise<void>((resolve) => { const oldConnectGate = new Promise<void>((resolve) => {
+6 -3
View File
@@ -612,13 +612,13 @@ describe("embedded SFU browser transport", () => {
expect(pc.transceivers[1]!.sender.parameters.encodings[0]!.maxBitrate).toBe(64_000); expect(pc.transceivers[1]!.sender.parameters.encodings[0]!.maxBitrate).toBe(64_000);
}); });
it("closes every owned clone during in-flight replacement without stopping the sources", async () => { it.each(["replaceTrack", "setParameters"] as const)("closes every owned clone while %s remains pending without stopping the sources", async (operation) => {
const { publisher: host, pc, video, sound } = await publisher(); const { publisher: host, pc, video, sound } = await publisher();
let release!: () => void; let release!: () => void;
const blocked = new Promise<void>((resolve) => { const blocked = new Promise<void>((resolve) => {
release = resolve; release = resolve;
}); });
pc.transceivers[0]!.sender.setParameters.mockImplementationOnce( pc.transceivers[0]!.sender[operation].mockImplementationOnce(
async () => blocked, async () => blocked,
); );
const replacement = new FakeTrack("video"); const replacement = new FakeTrack("video");
@@ -627,7 +627,6 @@ describe("embedded SFU browser transport", () => {
host.setPaused(true); host.setPaused(true);
expect(replacement.clones[0]!.enabled).toBe(false); expect(replacement.clones[0]!.enabled).toBe(false);
await host.disconnect(); await host.disconnect();
release();
expect(await replacing).toBe(false); expect(await replacing).toBe(false);
for (const track of [ for (const track of [
video.clones[0]!, video.clones[0]!,
@@ -638,6 +637,10 @@ describe("embedded SFU browser transport", () => {
for (const source of [video, sound, replacement]) for (const source of [video, sound, replacement])
expect(source.stop).not.toHaveBeenCalled(); expect(source.stop).not.toHaveBeenCalled();
expect(pc.close).toHaveBeenCalledOnce(); expect(pc.close).toHaveBeenCalledOnce();
const audioCalls = pc.transceivers[1]!.sender.replaceTrack.mock.calls.length;
release();
await vi.advanceTimersByTimeAsync(0);
expect(pc.transceivers[1]!.sender.replaceTrack).toHaveBeenCalledTimes(audioCalls);
}); });
it("fails closed when restoring a replaced sender fails", async () => { it("fails closed when restoring a replaced sender fails", async () => {
+131 -1
View File
@@ -1,8 +1,9 @@
import { describe, expect, it } from "vitest"; import { afterEach, describe, expect, it, vi } from "vitest";
import { import {
h264ProbeTarget, h264ProbeTarget,
h264ProbeSustainsTarget, h264ProbeSustainsTarget,
preferredVideoCodecForTrack,
type H264ProbeSample, type H264ProbeSample,
} from "../src/client/webrtc/video-codec-preflight.ts"; } from "../src/client/webrtc/video-codec-preflight.ts";
import { QUALITY_PROFILES } from "../src/client/media/quality.ts"; import { QUALITY_PROFILES } from "../src/client/media/quality.ts";
@@ -158,3 +159,132 @@ describe("H264 sender preflight", () => {
).toBe(false); ).toBe(false);
}); });
}); });
describe("codec probe lifetime", () => {
afterEach(() => {
vi.unstubAllGlobals();
vi.useRealTimers();
});
function fixture(blocked = "", failReceiver = false) {
vi.useFakeTimers();
vi.stubGlobal("window", globalThis);
let resolve!: () => void;
const pending = { promise: new Promise<void>((ready) => { resolve = ready; }), resolve: () => resolve() };
const wait = (stage: string) => blocked === stage ? pending.promise : Promise.resolve();
const source = { kind: "video", readyState: "live", getSettings: () => ({}), stop: vi.fn() };
const track = { ...source, stop: vi.fn() };
const createElement = vi.fn(() => ({
getContext: () => ({ fillRect() {} }),
captureStream: () => ({ getVideoTracks: () => [track] }),
}));
vi.stubGlobal("document", { createElement });
vi.stubGlobal("RTCRtpSender", {
getCapabilities: () => ({ codecs: [{ mimeType: "video/H264", clockRate: 90_000 }] }),
});
const connections: Connection[] = [];
class Connection {
constructor() {
if (failReceiver && connections.length === 1) throw new Error("no receiver");
connections.push(this);
}
close = vi.fn();
addEventListener() {}
localDescription = { type: "offer", sdp: "fixture" };
setLocalDescription = vi.fn(async () => {});
async setRemoteDescription() {}
async createOffer() { await wait("offer"); return this.localDescription; }
async createAnswer() { return { type: "answer", sdp: "fixture" }; }
addTransceiver() {
return {
setCodecPreferences() {},
sender: {
track,
getParameters: () => ({ encodings: [{}] }),
setParameters: () => wait("configure"),
getStats: async () => {
await wait("stats");
const timestamp = Date.now();
const frames = timestamp * 30 / 1_000;
return new Map([
["out", { id: "out", type: "outbound-rtp", kind: "video", timestamp,
codecId: "codec", mediaSourceId: "source", framesEncoded: frames }],
["codec", { type: "codec", mimeType: "video/H264" }],
["source", { id: "source", type: "media-source", frames }],
]);
},
},
};
}
}
vi.stubGlobal("RTCPeerConnection", Connection);
return {
source: source as unknown as MediaStreamTrack,
track, connections, createElement, pending,
};
}
it.each(["configure", "offer", "stats"])("bounds a pending %s operation and cleans up", async (stage) => {
const f = fixture(stage);
let result: string | undefined;
const run = preferredVideoCodecForTrack(f.source, QUALITY_PROFILES["1080p30"])
.then((codec) => { result = codec; });
await vi.advanceTimersByTimeAsync(3_999);
expect(result).toBeUndefined();
await vi.advanceTimersByTimeAsync(1);
expect(result).toBe("vp8");
await run;
expect(f.track.stop).toHaveBeenCalledOnce();
expect(f.source.stop).not.toHaveBeenCalled();
expect(f.connections.every((connection) => connection.close.mock.calls.length === 1)).toBe(true);
expect(vi.getTimerCount()).toBe(0);
f.pending.resolve();
await vi.advanceTimersByTimeAsync(0);
if (stage !== "stats") expect(f.connections[0]!.setLocalDescription).not.toHaveBeenCalled();
});
it("cancels pending SDP without waiting for the browser promise", async () => {
const f = fixture("offer");
const controller = new AbortController();
let result: string | undefined;
const run = preferredVideoCodecForTrack(f.source, QUALITY_PROFILES["1080p30"], controller.signal)
.then((codec) => { result = codec; });
await vi.advanceTimersByTimeAsync(0);
controller.abort();
await vi.advanceTimersByTimeAsync(0);
expect(result).toBe("vp8");
await run;
expect(f.track.stop).toHaveBeenCalledOnce();
expect(vi.getTimerCount()).toBe(0);
f.pending.resolve();
await vi.advanceTimersByTimeAsync(0);
expect(f.connections[0]!.setLocalDescription).not.toHaveBeenCalled();
});
it("does not allocate an already-cancelled probe", async () => {
const f = fixture();
await expect(preferredVideoCodecForTrack(f.source, QUALITY_PROFILES["1080p30"], AbortSignal.abort()))
.resolves.toBe("vp8");
expect(f.createElement).not.toHaveBeenCalled();
expect(f.connections).toHaveLength(0);
});
it("closes the sender if receiver construction fails", async () => {
const f = fixture("", true);
await expect(preferredVideoCodecForTrack(f.source, QUALITY_PROFILES["1080p30"]))
.resolves.toBe("vp8");
expect(f.connections[0]!.close).toHaveBeenCalledOnce();
expect(f.track.stop).toHaveBeenCalledOnce();
expect(vi.getTimerCount()).toBe(0);
});
it("keeps a proved H264 result and retires the budget", async () => {
const f = fixture();
const run = preferredVideoCodecForTrack(f.source, QUALITY_PROFILES["1080p30"]);
await vi.advanceTimersByTimeAsync(2_000);
await expect(run).resolves.toBe("h264");
expect(f.track.stop).toHaveBeenCalledOnce();
expect(f.source.stop).not.toHaveBeenCalled();
expect(vi.getTimerCount()).toBe(0);
});
});