test(media): add selected TURN allocation gate (#122)
Add a sanitized standalone Chrome allocation gate and retain the active Host-ingress selected TURN functional evidence. Ordinary peers remain STUN-only; initial ingress, peer-selected, mobile, resource, and performance gates remain open.
This commit is contained in:
1 parent
8dc929fc87
commit
c5729c53b4
10 files changed
+281
-38
No files matched your search
@@ -84,7 +84,9 @@ TURN config, issuer, capability, refresh wire, or client propagation. Stale
|
||||
`PEER_ICE_TURN_*` keys fail startup even when blank. The source candidate now
|
||||
implements a complete selected-edge tuple and one relay-only rebuild for either
|
||||
the initial pending Host publication or an active Host/SFU route; production
|
||||
configures the tuple, but real TURN/media evidence remains open.
|
||||
configures the tuple. A bounded exact-Web-source canary against the production
|
||||
LiveKit/coturn tuple proves the active Host/SFU relay route; initial Host ingress
|
||||
and `peer-selected` relay media remain open.
|
||||
The wire has two explicit edge kinds: `peer-selected` for the last-mile failed
|
||||
peer edge and `host-sfu-ingress` for a restricted Host-to-SFU retry. Ordinary
|
||||
Peer ICE remains STUN-only. Each room admits at most one pending or answered
|
||||
@@ -97,7 +99,7 @@ production. The tracked LiveKit sample explicitly sets `tcp_port: 0`, disables
|
||||
TCP fallback, supplies
|
||||
the deployment-owned STUN server, and configures no TURN service. The SFU controller still
|
||||
activates only after a peer edge exhausts recovery. Public participant entry has
|
||||
been observed, but retained media and route admission remain unverified. The old
|
||||
been observed, but public-room retained media and route admission remain unverified. The old
|
||||
`769de201f7cc` release and coturn relay are isolated rollback resources, not a
|
||||
permanent compatibility branch or advertised current transport.
|
||||
|
||||
@@ -431,9 +433,25 @@ relay-only LiveKit publisher `RTCConfiguration`, including during the initial
|
||||
pending prepare. Pinned LiveKit client 2.22.0 accepts `rtcConfig` on
|
||||
`Room.connect`; its engine clones that override before creating the publisher
|
||||
PeerConnection and does not replace explicitly supplied ICE servers. This path
|
||||
is configured in production but has no real relay-media evidence.
|
||||
is configured in production, and its active Host-ingress relay function is now
|
||||
proven; the initial-ingress and `peer-selected` variants remain open.
|
||||
LiveKit participant-wide embedded/external TURN remains a separate ICE domain.
|
||||
|
||||
On 2026-08-21 a standalone relay-only Chrome allocation gate issued a credential
|
||||
through the production parser/issuer and received one UDP relay candidate. One
|
||||
subsequent bounded Chrome 151 canary ran the exact deployed Web source in a local
|
||||
Screener room against the production LiveKit/coturn tuple. A temporary DEV-only
|
||||
hook forced the active SFU publisher to fail once and was removed afterward. The
|
||||
application then issued `host-sfu-ingress`, connected a relay-policy publisher
|
||||
whose only ICE server was TURN, selected a succeeded/nominated pair with
|
||||
`relayProtocol=udp`, sent video bytes and frames, and kept Viewer decode/render
|
||||
counters advancing. Ordinary peer PCs remained STUN-only, Host outbound media
|
||||
edges peaked at one, and explicit stop returned them to zero. This proves the
|
||||
active functional transition only; it is not latency, quality, capacity, expiry,
|
||||
mobile, or resource evidence. Chrome reported `candidateType=prflx` for the same
|
||||
local candidate; the standards-bounded interpretation and raw-counter summary
|
||||
are retained in the linked research rather than normalized away.
|
||||
|
||||
### Deferred Optimization
|
||||
|
||||
These items are not current runtime behavior:
|
||||
|
||||
+24
-8
@@ -35,9 +35,12 @@ and the old endpoint remains 404. Exact
|
||||
`22119b907d3cf03ce8b06d6fb4596ce1a26fedd7`
|
||||
is the immediate rollback target; exact `fd76277b05d491af8840b28f3132b7ff445d3cbe`
|
||||
remains a secondary rollback.
|
||||
The selected-edge UDP tuple is configured with TTL 120, but no real TURN/SFU
|
||||
media canary was triggered; this remains a bounded configuration smoke, not
|
||||
broad-rollout media acceptance.
|
||||
The selected-edge UDP tuple is configured with TTL 120. The cutover itself
|
||||
triggered no TURN/SFU media and remains configuration evidence. A separate
|
||||
post-deploy local Screener canary using this exact Web source and the production
|
||||
LiveKit/coturn tuple later proved active Host-ingress relay function without
|
||||
using the production application room or database; it is not broad-rollout or
|
||||
performance acceptance.
|
||||
|
||||
## Topology and prerequisites
|
||||
|
||||
@@ -213,6 +216,16 @@ receive a short-lived grant. Roll back by removing the application tuple before
|
||||
changing coturn or firewall state. Credentials never enter URLs, logs, browser
|
||||
persistence, room rows, or SQLite.
|
||||
|
||||
Before a media canary, `npm run gate:turn-allocation` provides the bounded
|
||||
allocation check. Set `CHROME_PATH` in the process environment and supply the
|
||||
complete selected-edge tuple either in that environment or as newline-delimited
|
||||
`KEY=value` records on stdin. Values containing `=` are preserved. The script
|
||||
uses an isolated Chrome profile and loopback-only in-memory credential response,
|
||||
then emits only status, relay-candidate count, transport protocol, and coarse ICE
|
||||
error-code buckets. Do not echo, log, or persist raw production environment
|
||||
input; never add TURN URLs, usernames, credentials, candidate addresses, or raw
|
||||
errors to the result.
|
||||
|
||||
`PEER_ASSISTED_ROOM_IDS` is retired. Supplying it, even blank, fails startup so
|
||||
that a stale room-1 deployment cannot silently retain the old scope. With
|
||||
`PEER_ASSISTED_MEDIA=true`, every normal room receives peer-assisted routing,
|
||||
@@ -515,7 +528,8 @@ active/running with their observed `NRestarts=0`. Backup
|
||||
`/opt/screener/backups/16f6eab27bdf-precutover-20260820T234457Z` retains the
|
||||
prior environment, database, and exact current target. This deploys the one-use
|
||||
initial/active Host-ingress selected retry; no browser, SFU media, TURN relay,
|
||||
or performance canary ran, so TURN transport remains unproved.
|
||||
or performance canary ran during that cutover. The later post-deploy functional
|
||||
canary is recorded above and does not change this cutover record.
|
||||
|
||||
Enabling persistence does not migrate rooms that existed only in memory. The
|
||||
deployment restart invalidates those temporary links; the first subsequently
|
||||
@@ -632,7 +646,8 @@ state. That host retains an older authenticated-relay configuration and TCP/UDP
|
||||
3478 plus UDP 49152-49251 firewall range. In this template Screener advertises
|
||||
no TURN credential and the canary audit found zero allocations; production
|
||||
selected-edge issuance remains limited to the controller's one configured edge
|
||||
and still lacks a real forced-relay media acceptance.
|
||||
and has one active Host-ingress forced-relay functional acceptance. Initial
|
||||
Host ingress and `peer-selected` last mile remain open.
|
||||
|
||||
Copy [`deploy/coturn/turnserver.conf.example`](../deploy/coturn/turnserver.conf.example)
|
||||
to an untracked service-owned location and use the tracked
|
||||
@@ -674,9 +689,10 @@ Run these checks from real external networks before calling the deployment usabl
|
||||
3. On a normal room, exhaust a peer route and verify the host plus at
|
||||
most two necessary roots select LiveKit UDP 7882. Peer descendants stay on
|
||||
ordinary direct UDP and host/relay downstream caps remain two/one.
|
||||
4. Block all UDP on one test client. Verify bounded ICE recovery ends in a clear
|
||||
connection failure, without an ICE/TCP, TURN/TCP, or long pseudo-connected
|
||||
path. No force-relay product branch exists in this candidate.
|
||||
4. Force one controller-eligible edge past SFU/UDP and verify only that edge gets
|
||||
the short-lived TURN server plus relay policy and selects TURN/UDP. Ordinary
|
||||
peer PCs must stay STUN-only. Then block all UDP and verify bounded recovery
|
||||
ends clearly without ICE/TCP, TURN/TCP, or a long pseudo-connected path.
|
||||
5. Exercise root departure, reconnect, SFU unavailable, route prepare rollback,
|
||||
stop, and source/profile changes. Unaffected subtrees must not migrate.
|
||||
6. Repeat at 1, 3, 5, and 8 viewers across representative consumer networks.
|
||||
|
||||
@@ -25,7 +25,7 @@ Last updated: 2026-08-21
|
||||
- Keep experiments bounded: the standard representation sequence is below; native RTP relay, encoded-object striping, and FEC remain separate.
|
||||
- ADR-0006's historical browser-bridge canary remains no-go; VP8 stays default. Browser H.264 rendered 298/299; Native WGC/MF rendered 203 with PID/LUID-correlated `VideoEncode`. Downloaded use, multi-viewer/endurance/public proof remain open.
|
||||
- Mobile endpoints are Web Viewer-only. Current Android/iOS browsers do not expose Web Host capture; AirPlay/system mirroring is Viewer-local output.
|
||||
- ADR-0005 accepts direct/peer UDP, bounded SFU roots, then selected-edge TURN. Source admits one pending or answered `peer-selected` attempt per room while `host-sfu-ingress` remains independent. Production grants one revision-bound, one-use retry for initial/active Host ingress. The deployed tuple/control path have no production SFU/TURN media canary; participant-wide TURN stays rejected and ordinary peers stay STUN-only.
|
||||
- ADR-0005: direct/peer UDP -> bounded SFU roots -> selected-edge TURN. Source caps pending/answered `peer-selected` at one per room; Host ingress is independent. An exact-source/production-media canary proves active Host ingress only. Ordinary peers stay STUN-only.
|
||||
- The controller is process-enabled, not room-allowlisted: `PEER_ASSISTED_MEDIA=true` gives every normal room the same direct/peer -> SFU -> selected-edge path with per-room state. Room `1` is historical smoke. Preserve sticky P2P, mobile leaves and break-before-make; recovery gets one attempt per layer (ICE restart, same-parent rebuild, alternate peer, then SFU), never three identical retries.
|
||||
- C+B quality reparenting is edge-local and cooldown-bound; no score, timer, or global parent penalty.
|
||||
- Flagship media is UDP; HTTPS/WSS stays TLS/TCP; the old release remains rollback-only.
|
||||
@@ -45,13 +45,13 @@ Last updated: 2026-08-21
|
||||
- Production is exact `16f6eab27bdfb1c15cdbd814a35864f4f18be767` at `https://share.bonfire.icu`; its 1,027,423-byte artifact SHA-256 is `74e0274ab11a162cb9dd4be1c34bb6b639e2ea76005969c30ae3a1daa656742f`. The 2026-08-20T23:44:57Z cutover took 10,238 ms lock/641 ms stop-health/570 ms switch-health. Immediate rollback is exact `22119b907d3cf03ce8b06d6fb4596ce1a26fedd7`; exact `fd76277b05d491af8840b28f3132b7ff445d3cbe` remains secondary.
|
||||
- `main` is ahead only in Native/test evidence; the Windows Native sender stays source-only. Host-ingress retry is deployed.
|
||||
- The four services are active/running with observed `NRestarts=0`; local/public health are 200. SQLite v3 has five rooms including room `1`, with DB owner/mode `screener:screener`/0600.
|
||||
- Production ICE remains ordinary STUN-only. `PEER_ASSISTED_MEDIA=true` enables the controller for every normal room, and selected-edge TURN is configured for the single UDP tuple with TTL 120; the coturn daemon and nginx/LiveKit/firewall/listener baselines were unchanged. No production SFU/TURN media canary or quality claim is implied.
|
||||
- Production ICE is ordinary STUN-only. The all-room controller has one selected-edge UDP tuple at TTL 120. A local exact-source canary used production LiveKit/coturn without production app/DB writes and proved active Host-ingress function only.
|
||||
- Production requires the independent site-access secret through `SITE_ACCESS_PASSWORD`. Its stateless cookie authorizes creation/Host and permits code-only Viewer attempts; valid private fragment grants remain direct. Public-watch accepts site access plus code, while private code-only entry additionally requires the room password. The env, endpoint, cookie, and nginx limiter naming migrated atomically; anonymous failures stay neutral, and there are no accounts/JWT/session rows.
|
||||
- Production uses nginx, Node.js 24.19.0, coturn 4.17.2, and LiveKit 1.13.5 on UDP 7882 (TCP fallback off). Ordinary peer ICE receives no TURN; the selected-edge tuple is configured only for the current controller edge, with no credential pre-advertised to ordinary peers. Coturn retains the old authenticated-relay ports. Services are healthy with zero restarts.
|
||||
- ADR-0005 rejects `PEER_ASSISTED_ROOM_IDS`; `PEER_ASSISTED_MEDIA=true` enables every room and ordinary peers stay STUN-only. Initial or active Host ingress may consume one bound relay-only grant; ready/abort clears it and failure restores peer baseline. Stale room-ID config fails startup; `screener-v2` is the only deployed wire.
|
||||
- Chrome 151/LiveKit localhost A/B cut failure-to-active/render from 1.481/2.257 seconds to 0.200/0.320; 31 new frames and 25 ms sampling kept host edges at two. It is headless synthetic 720p30 and includes SDK/network prewarm, not public-network evidence.
|
||||
- Chrome 151 synthetic topology/quality runs kept fanout 2/1 and decoding; one relay close recovered in 5.32 seconds. Control evidence only.
|
||||
- Deployed Web source uses LiveKit 2.22.0's two-layer `q,h` layout. Local Chrome 151/LiveKit 1.13.5 kept one SFU/UDP root with growing inbound/decoded/rendered counters, clean leaves and no TURN. Production SFU media, zero-child BWE, and calibrated C+B loss remain open.
|
||||
- Deployed Web uses LiveKit 2.22.0 `q,h`. Chrome 151 proves local SFU/UDP and, separately, active selected TURN/UDP Host ingress against production media services with Viewer frame progress and clean stop. Initial/peer-selected relay, BWE, and C+B remain open.
|
||||
- C+B uses three hard-bad pairs, one-use samples and guarded intent/cooldown. Deployed admission rescue moves the oldest childless zero-capacity Host leaf below an unassigned one-slot relay in one revision; no score/timer/global rebalance. Deployment passed, but no real room triggered it.
|
||||
- Web names/window audio are deployed; source adds full-display system audio, music hint, interval AV loss% and audio codec/bitrate/jitter details. Native WGC/MF remains source-only: 203 rendered frames, 500 Opus packets, matched PID/LUID `VideoEncode`. New Web audio work is undeployed; VP8 remains default.
|
||||
|
||||
@@ -68,7 +68,7 @@ These are measurement gates, not performance claims.
|
||||
|
||||
- Sustainable count by hardware, quality, network, and route: finish instrumented `1/3/5/8`, then pass a 20-viewer matrix before changing the accepted target default to 20.
|
||||
- Whether ADR-0004 passes fanout, re-encoding, depth-four latency, reparenting, silent-partition, and mobile-leaf gates.
|
||||
- Whether selected-edge passes forced relay, expiry/failure, mobile, 1-GiB resource and bandwidth gates. Coturn's bearer is non-revocable until expiry; application guards and TTL only bound reuse. Performance comparison and the retained matrix follow functional landing; Media TCP is out of scope.
|
||||
- Selected-edge: active Host-ingress function passes; initial ingress, `peer-selected`, expiry/failure, mobile, resource and bandwidth gates remain. The bearer is non-revocable until expiry; performance follows later and Media TCP is out.
|
||||
- Exact mobile Viewer lifecycle behavior across autoplay, rotation, backgrounding and network changes.
|
||||
- Project license and distribution model, which determines whether GPL/AGPL sources can move beyond study-only use.
|
||||
- Initial deployment regions and expected mainland China, Hong Kong, and overseas network mix.
|
||||
|
||||
@@ -1,14 +1,15 @@
|
||||
# Low-Server-Cost Media Routes
|
||||
|
||||
- Research date: 2026-08-19
|
||||
- Research date: 2026-08-21
|
||||
- Scope: one broadcaster, at most eight trusted viewers, low latency, and host
|
||||
media fanout at most two
|
||||
- Status: ADR-0005 accepts STUN-only direct/peer UDP, bounded SFU/UDP roots,
|
||||
then optional authenticated TURN/UDP for one controller-selected exceptional
|
||||
edge. Production enables the controller for all rooms and has a configured
|
||||
selected-edge TURN tuple; ordinary peer connections remain STUN-only, while
|
||||
production SFU/TURN media remains unverified. Latest source passed one local
|
||||
SFU/UDP functional path; TURN remains unverified.
|
||||
selected-edge TURN tuple; ordinary peer connections remain STUN-only. A
|
||||
post-deploy exact-Web-source canary against the production LiveKit/coturn tuple
|
||||
passed active Host SFU -> selected TURN/UDP ingress. Initial ingress,
|
||||
peer-selected last mile, external cohorts, and performance remain unverified.
|
||||
|
||||
## Current Route Ladder
|
||||
|
||||
@@ -44,9 +45,10 @@ Chrome 151, LiveKit 1.13.5, and pinned client 2.22.0. LiveKit produced two sende
|
||||
RIDs in `q,h` order; Screener's former `q,f` guard caused the Host publisher to
|
||||
fail closed at `sender-config`. After correcting that contract, Viewer inbound
|
||||
packets and decoded/rendered frames increased, endpoint edges stayed bounded,
|
||||
and both clients left cleanly. TURN did not participate. This is functional,
|
||||
not performance, evidence; production/public transport and browser validation
|
||||
remain open.
|
||||
and both clients left cleanly. TURN did not participate in that first run. A
|
||||
later exact-source canary against the production media tuple proved the active
|
||||
Host-ingress selected relay described below. Both are functional, not
|
||||
performance, evidence; public-room and browser-diversity validation remain open.
|
||||
|
||||
## Token-Free SFU Standby Prewarm
|
||||
|
||||
@@ -207,20 +209,51 @@ without advertising that credential to ordinary peer PCs.
|
||||
Ordinary peer ICE remains STUN-only. The participant-wide TURN
|
||||
config, capability and refresh wire are removed; any stale `PEER_ICE_TURN_*`
|
||||
key, including an empty value, fails startup. Selected-edge TURN is deployed as
|
||||
a configured, controller-issued exceptional transport; its real media path is
|
||||
not yet canary-proven. Every ordinary Web peer and Native-shaped client remains
|
||||
STUN-only. The tracked coturn example is UDP
|
||||
a configured, controller-issued exceptional transport. Its active Host-ingress
|
||||
media path is canary-proven; initial ingress and `peer-selected` remain open.
|
||||
Every ordinary Web peer and Native-shaped client remains STUN-only. The tracked coturn example is UDP
|
||||
`stun-only`; the LiveKit example exposes only ICE/UDP mux 7882, explicitly sets
|
||||
`tcp_port: 0` and `allow_tcp_fallback: false`, supplies the self-hosted STUN
|
||||
endpoint, and configures no external or embedded TURN. Candidate validation and rollback use isolated
|
||||
instances rather than a process-wide old-release compatibility branch. HTTPS/WSS remains TLS/TCP.
|
||||
|
||||
### Active Host-Ingress Functional Evidence
|
||||
|
||||
On 2026-08-21 the tracked standalone gate reused the production selected-edge
|
||||
tuple parser and credential issuer, supplied the short-lived credential only to
|
||||
an isolated loopback Chrome page, and returned a sanitized result: one UDP relay
|
||||
candidate and an otherwise non-blocking `701` error bucket. No URL, username,
|
||||
credential, candidate address, or raw error text was emitted or persisted.
|
||||
|
||||
The canonical Chrome 151 product canary then used exact deployed Web source
|
||||
`16f6eab27bdfb1c15cdbd814a35864f4f18be767` in a local Screener room while
|
||||
connecting to the production LiveKit 1.13.5 and coturn tuple. After direct media
|
||||
and active SFU media, a temporary DEV-only hook failed the refreshed publisher
|
||||
once; the normal application failure path requested `host-sfu-ingress` and the
|
||||
hook was removed after the run. The relay-policy publisher had only a TURN
|
||||
server, reached connected/ICE-connected, and selected a succeeded/nominated UDP
|
||||
pair with `relayProtocol=udp`. Pair counters reached 1,054,606 bytes sent and
|
||||
4,934 received; outbound RTP reached 837,103 bytes and 101 frames. Viewer decode
|
||||
and render counters both advanced during the selected stage. Ordinary peer PCs
|
||||
kept policy `all` with STUN-only servers, Host active outbound edges stayed at
|
||||
one, and explicit stop reduced them to zero. The production Screener app and
|
||||
database were not used or modified.
|
||||
|
||||
Chrome reported the selected local candidate's `candidateType` as `prflx` while
|
||||
also reporting `relayProtocol=udp`. That tuple is internally inconsistent with
|
||||
the W3C model: `iceTransportPolicy="relay"` permits only media-relay candidates,
|
||||
and `relayProtocol` is defined for a local relay candidate. The evidence is
|
||||
therefore retained verbatim and classified from the relay-only configuration,
|
||||
TURN-only server set, `relayProtocol`, and transmitted media together; the
|
||||
`candidateType` field is not rewritten. This proves one active functional route,
|
||||
not latency, quality, capacity, expiry, mobile, resource, or bandwidth behavior.
|
||||
Coturn 4.17.2 documents `stun-only` as ignoring TURN requests and provides
|
||||
`no-tcp` and `no-tls`; it marks `no-dtls` deprecated, so the tracked temporary
|
||||
template does not use that switch. The shared production host instead retains its old
|
||||
authenticated-relay daemon and TCP/UDP 3478 plus UDP 49152-49251 rules. The
|
||||
application advertises no TURN credential, and the post-canary audit found zero
|
||||
allocations. This baseline is neither the accepted selected-edge rollout nor
|
||||
proof that relay media works.
|
||||
rejected participant-wide candidate advertised no TURN credential, and that
|
||||
historical post-canary audit found zero allocations. This baseline is not the
|
||||
later selected-edge functional evidence above.
|
||||
|
||||
No public port is selected by this decision. LiveKit documents ICE/UDP mux as
|
||||
optional and its pinned sample recommends a multi-port UDP mux range at least
|
||||
@@ -300,7 +333,7 @@ lost.
|
||||
| Fixed two-chain browser relay | Host emits at most two copies; each relay emits at most one | Ordinary browser, but every relay decodes and re-encodes and adds a hop | Current default-off, maximum-eight-viewer spike |
|
||||
| Native shared-encode host | Host targets one encode for at most two standard WebRTC edges | libwebrtc public-API proxy risk spike, with Pion as fallback | Planned separate sender phase; still pays per-edge upload |
|
||||
| Native volunteer encoded-RTP relay | Each volunteer forwards one encoded copy | Native install, RTP/RTCP forwarding, packaging, and opt-in relay policy | Conditional experiment only if relay re-encoding is the sole browser-spike failure |
|
||||
| SFU virtual parent | SFU normally emits one or two root copies; roots keep peer descendants | Service pays measured root egress; host sends one publication | Accepted primary central fallback after direct/peer UDP; latest source passes one local SFU/UDP root, while production evidence remains open |
|
||||
| SFU virtual parent | SFU normally emits one or two root copies; roots keep peer descendants | Service pays measured root egress; host sends one publication | Accepted primary central fallback after direct/peer UDP; local SFU/UDP and active Host-ingress relay function pass, while public-room/performance evidence remains open |
|
||||
| Exceptional server-fed viewers | SFU/TURN emits necessary copies that no healthy root can distribute | Additional capped central egress | Explicit compatibility exception only; never unbounded whole-room fanout |
|
||||
| SVC plus multiple trees | Peers emit striped layer copies across several trees | Layer scheduling, reassembly, redundancy, and more churn state | Separate conditional spike; target endpoint upload near `B` |
|
||||
| Network coding | Peers or servers emit coded blocks | Generations, buffering, decoding, integrity, and a custom media plane | Trace/FEC spike only; optimize loss recovery, not clean bandwidth |
|
||||
@@ -387,8 +420,8 @@ Primary sources checked on 2026-08-19 and 2026-08-21:
|
||||
- [WebRTC transports, RFC 8835](https://www.rfc-editor.org/rfc/rfc8835.html) -
|
||||
browser transport capability requirements do not require an application to
|
||||
advertise every supported fallback on every connection.
|
||||
- [WebRTC](https://www.w3.org/TR/webrtc/),
|
||||
[WebRTC Statistics](https://www.w3.org/TR/webrtc-stats/),
|
||||
- [WebRTC](https://w3c.github.io/webrtc-pc/),
|
||||
[WebRTC Statistics](https://w3c.github.io/webrtc-stats/),
|
||||
[WebRTC Encoded Transform](https://w3c.github.io/webrtc-encoded-transform/#stream-processing),
|
||||
and [WebRTC SVC](https://www.w3.org/TR/webrtc-svc/) - W3C specifications; SVC
|
||||
is an encoding control, not a distribution topology.
|
||||
|
||||
+6
-6
@@ -6,7 +6,7 @@ Last updated: 2026-08-21
|
||||
|
||||
Production at `https://share.bonfire.icu` is exact `16f6eab27bdfb1c15cdbd814a35864f4f18be767`; artifact 1,027,423 bytes, SHA-256 `74e0274ab11a162cb9dd4be1c34bb6b639e2ea76005969c30ae3a1daa656742f`.
|
||||
Cutover: 10,238 ms lock/641 ms stop-health/570 ms switch-health; SQLite v3/five rooms/four active services, observed `NRestarts=0`. Initial Host-ingress retry is deployed; mobile endpoints are Web Viewer-only.
|
||||
Ordinary ICE is STUN-only; all rooms use the bounded peer/SFU-UDP controller and selected-edge UDP TTL 120. Room `1` is historical smoke; production SFU/TURN media remains unverified.
|
||||
Ordinary ICE is STUN-only; all rooms use bounded peer/SFU-UDP plus selected-edge UDP TTL 120. Room `1` is historical. An exact-source/production-media canary proves active Host-ingress relay only.
|
||||
Web names/audio hint are deployed. Native WGC/MF/package is source-only; Web defaults VP8.
|
||||
|
||||
## Execution Principle
|
||||
@@ -21,7 +21,7 @@ Flagship; parallel; min run/smoke/rollback; benchmark later. Active UI/config/lo
|
||||
- Native v2 is source-only (memory rooms, 300s reclaim); it follows authenticated direct-child assignments up to two and fails unsupported SFU/selected ingress boundedly.
|
||||
- PR #44's STUN-only/SFU-UDP router and token-free fallback prewarm are process-enabled for all normal rooms: roots <=2, browser relay <=1, bounded failure. Room `1` is historical smoke; mobile/iPad viewers are leaves, healthy edges sticky, and HTTPS/WSS stays TLS/TCP.
|
||||
- After an answer, a generation-bound 15s initial-connect deadline enters ICE restart; success/replacement/disposal cancels it. It is deployed but not mobile-verified.
|
||||
- Production includes LiveKit's two-layer `q,h` sender guard; its functional media proof remains local, not production.
|
||||
- Production `q,h` is deployed; an exact-source/production-media canary proves active Host ingress without production app/DB.
|
||||
- Room `1` C+B reparenting remains uncalibrated. Deployed admission rescue promotes an unassigned one-slot relay over the oldest childless zero-capacity Host leaf, with no scores or periodic optimization.
|
||||
- Web roster names are deployed. Source window/system audio, music hint, interval AV loss% and audio codec/bitrate/jitter details await deploy.
|
||||
|
||||
@@ -36,13 +36,13 @@ Flagship; parallel; min run/smoke/rollback; benchmark later. Active UI/config/lo
|
||||
- Native Win11 audio is source-only/default-off: target isolation was 4018x and one Viewer got 495 Opus packets. Download, game sync, Win10, and other routes remain open.
|
||||
- Access protocol/config/HTTP/storage/SQLite/signaling focused tests pass, including commit-first teardown and v1 rollback.
|
||||
- Native wire/session tests enforce at most two authoritative children, no presence-created edge, stale-revision ignore, and one bounded unsupported-route failure per revision.
|
||||
- Production includes initial/active selected Host ingress and peer last-mile; source caps pending plus answered `peer-selected` attempts at one per room without counting Host ingress. Focused tests cover admission, release, independent ingress, exactly-once retry, cleanup, rollback, and ordinary STUN-only. No relay media ran.
|
||||
- Source caps pending/answered `peer-selected` at one per room, excluding Host ingress. Focused tests cover admission/release/rollback/STUN-only; exact-source Chrome proves active TURN/UDP media, Viewer progress, Host edge one and stop zero. Functional only.
|
||||
- Host names/window audio/all-room routing/Host selected ingress are active. New AV stats pass 42 focused tests/typecheck; this is not audible proof.
|
||||
- Exact `16f6eab` passed typecheck, both builds and 30 files/445 tests. Deploy preserved all recorded config/network/SQLite hashes; DB, routes, services, endpoints, site access, and old-endpoint 404 passed. It triggered no TURN/SFU session. Rollback is exact `22119b907d3cf03ce8b06d6fb4596ce1a26fedd7`; `fd76277b05d491af8840b28f3132b7ff445d3cbe` remains secondary.
|
||||
|
||||
## Unverified Boundaries
|
||||
|
||||
- The `16f6eab` deploy proves process/config health only; production SFU frames, initial/active retry/failback, relay pair, admission rescue, caps, quality, and resource evidence remain open. Game-share load/blur is unclassified.
|
||||
- The cutover is config evidence; the separate canary closes active Host-ingress relay/frame/cap/stop function. Initial ingress, peer last mile, external cohorts, admission rescue, quality/resources/performance, and game blur remain open.
|
||||
- Mobile remains Viewer leaf-only. AirPlay/system mirroring is local output; live WebRTC `srcObject` has no portable in-app TV-output contract.
|
||||
- Silent partitions can wait 30 to 60 seconds for heartbeat detection before the default 5-second grace; this remains unverified.
|
||||
- The former release reportedly sustained bandwidth-limited blur on a capable LAN; Host refresh recovered while Viewer refresh did not. The new deployment has not yet reproduced or cleared it.
|
||||
@@ -54,10 +54,10 @@ Flagship; parallel; min run/smoke/rollback; benchmark later. Active UI/config/lo
|
||||
|
||||
## Next Milestone
|
||||
|
||||
Run one bounded production room `1` canary that retains UDP, a rendered frame, edge caps, and clean stop. TURN/performance follow later.
|
||||
Next: representative production-room UDP/frame/cap/stop canary; room `1` is history. TURN performance follows later.
|
||||
Recovery uses one ICE restart, one same-parent rebuild, one alternate peer, then SFU. Next gate exactly-two/Dynacast-off BWE on a zero-child SFU leaf, then root evacuation.
|
||||
Native WGC/MF passed one-Viewer hardware; package download, Viewer2/FIFO and game A/V remain. Benchmark later.
|
||||
Ordinary peers stay STUN-only. Initial/active Host ingress needs forced-relay canaries before any media claim.
|
||||
Ordinary peers stay STUN-only; initial ingress and `peer-selected` still need forced-relay proof.
|
||||
ADR-0004 still needs the 30-minute `1/3/5/8` matrix; a separate 20-viewer gate must pass before changing the default eight.
|
||||
|
||||
## Blockers And Decisions
|
||||
|
||||
@@ -14,6 +14,7 @@
|
||||
"build:server": "tsc -p tsconfig.server.json",
|
||||
"start": "node dist/server/server/index.js",
|
||||
"benchmark:peer-assisted": "tsx scripts/peer-assisted-benchmark.ts",
|
||||
"gate:turn-allocation": "tsx scripts/turn-allocation-gate.ts",
|
||||
"gate:native-one-viewer": "tsx scripts/native-one-viewer-gate.ts",
|
||||
"typecheck": "tsc -b --pretty false",
|
||||
"test": "vitest run",
|
||||
|
||||
@@ -0,0 +1,126 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { once } from "node:events";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { mkdtemp, rm } from "node:fs/promises";
|
||||
import { createServer } from "node:http";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { pathToFileURL } from "node:url";
|
||||
|
||||
import { parseSelectedEdgeTurn } from "../src/server/config";
|
||||
import { issueSelectedEdgeTurnCredential } from "../src/server/selected-edge-turn";
|
||||
|
||||
const allowedProtocols = new Set(["udp", "tcp", "unknown"]);
|
||||
const allowedErrorBuckets = new Set(["3xx", "4xx", "5xx", "6xx", "701", "other"]);
|
||||
|
||||
export interface TurnAllocationResult {
|
||||
status: "passed" | "failed";
|
||||
relayCandidateCount: number;
|
||||
protocols: string[];
|
||||
errorCodeBuckets: string[];
|
||||
}
|
||||
|
||||
export function parseGateEnvironment(input: string): NodeJS.ProcessEnv {
|
||||
const environment: NodeJS.ProcessEnv = {};
|
||||
for (const line of input.split(/\r?\n/)) {
|
||||
const separator = line.indexOf("=");
|
||||
if (separator > 0) environment[line.slice(0, separator)] = line.slice(separator + 1);
|
||||
}
|
||||
return environment;
|
||||
}
|
||||
|
||||
export function parseGateResult(input: string): TurnAllocationResult {
|
||||
let value: unknown;
|
||||
try { value = JSON.parse(input); } catch { throw new Error("Chrome returned an invalid TURN allocation result"); }
|
||||
if (!value || typeof value !== "object") throw new Error("Chrome returned an invalid TURN allocation result");
|
||||
const raw = value as Record<string, unknown>;
|
||||
if ((raw.status !== "passed" && raw.status !== "failed") ||
|
||||
!Number.isInteger(raw.relayCandidateCount) || Number(raw.relayCandidateCount) < 0 ||
|
||||
!Array.isArray(raw.protocols) || !raw.protocols.every((item) => allowedProtocols.has(item)) ||
|
||||
!Array.isArray(raw.errorCodeBuckets) || !raw.errorCodeBuckets.every((item) => allowedErrorBuckets.has(item))) {
|
||||
throw new Error("Chrome returned an invalid TURN allocation result");
|
||||
}
|
||||
const protocols = [...new Set(raw.protocols as string[])];
|
||||
if (raw.status === "passed" && (Number(raw.relayCandidateCount) === 0 ||
|
||||
protocols.length !== 1 || protocols[0] !== "udp")) {
|
||||
throw new Error("Chrome returned an invalid TURN allocation result");
|
||||
}
|
||||
return { status: raw.status, relayCandidateCount: Number(raw.relayCandidateCount),
|
||||
protocols,
|
||||
errorCodeBuckets: [...new Set(raw.errorCodeBuckets as string[])] };
|
||||
}
|
||||
|
||||
const page = `<!doctype html><script>
|
||||
(async()=>{const errors=new Set(),protocols=new Set();let count=0,done=false,pc;
|
||||
const finish=async status=>{if(done)return;done=true;clearTimeout(timer);pc?.close();await fetch("/result",{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({status,relayCandidateCount:count,protocols:[...protocols],errorCodeBuckets:[...errors]})})};
|
||||
const bucket=n=>n===701?"701":n>=300&&n<700?Math.floor(n/100)+"xx":"other";
|
||||
const timer=setTimeout(()=>finish("failed"),10000);
|
||||
try{const iceServer=await fetch("/credential",{cache:"no-store"}).then(r=>r.json());pc=new RTCPeerConnection({iceServers:[iceServer],iceTransportPolicy:"relay"});
|
||||
pc.onicecandidateerror=e=>errors.add(bucket(e.errorCode));pc.onicecandidate=e=>{if(e.candidate){if(e.candidate.type==="relay"){count++;protocols.add(e.candidate.protocol||"unknown")}}else finish(count>0&&protocols.size===1&&protocols.has("udp")?"passed":"failed")};
|
||||
pc.createDataChannel("allocation");await pc.setLocalDescription(await pc.createOffer())}catch{finish("failed")}})();
|
||||
</script>`;
|
||||
|
||||
export async function main(): Promise<number> {
|
||||
const chromePath = process.env.CHROME_PATH?.trim();
|
||||
if (!chromePath) throw new Error("CHROME_PATH is required");
|
||||
const stdin = process.stdin.isTTY ? "" : readFileSync(0, "utf8");
|
||||
const config = parseSelectedEdgeTurn({ ...parseGateEnvironment(stdin), ...process.env });
|
||||
if (!config) throw new Error("The selected-edge TURN tuple is required");
|
||||
const grant = issueSelectedEdgeTurnCredential(config, { edgeKind: "host-sfu-ingress",
|
||||
roomId: "allocation-gate", shareGeneration: "gate-share", revision: 1,
|
||||
hostPeerId: "gate-host", hostSessionId: "gate-session",
|
||||
publicationGeneration: "gate-publication", oldConnectionId: "gate-publication",
|
||||
newConnectionId: "gate-connection" }, Date.now());
|
||||
let settle!: (result: TurnAllocationResult) => void;
|
||||
const resultPromise = new Promise<TurnAllocationResult>((resolve) => { settle = resolve; });
|
||||
const server = createServer((request, response) => {
|
||||
response.setHeader("Cache-Control", "no-store");
|
||||
if (request.url === "/credential") response.end(JSON.stringify(grant.iceServer));
|
||||
else if (request.url === "/") response.end(page);
|
||||
else if (request.url === "/result" && request.method === "POST") {
|
||||
let body = "";
|
||||
request.setEncoding("utf8");
|
||||
request.on("data", (chunk) => { if (body.length < 4_096) body += chunk; });
|
||||
request.on("end", () => {
|
||||
try { settle(parseGateResult(body)); response.end(); }
|
||||
catch { response.statusCode = 400; response.end(); }
|
||||
});
|
||||
}
|
||||
else { response.statusCode = 404; response.end(); }
|
||||
});
|
||||
const profile = await mkdtemp(join(tmpdir(), "screener-turn-gate-"));
|
||||
let chrome: ReturnType<typeof execFile> | undefined;
|
||||
let chromeClosed: Promise<void> = Promise.resolve();
|
||||
let timeout: NodeJS.Timeout | undefined;
|
||||
try {
|
||||
server.listen(0, "127.0.0.1"); await once(server, "listening");
|
||||
const address = server.address();
|
||||
if (!address || typeof address === "string") throw new Error("TURN gate listener failed");
|
||||
chrome = execFile(chromePath, ["--headless=new", "--no-first-run",
|
||||
"--no-default-browser-check", "--disable-extensions", `--user-data-dir=${profile}`,
|
||||
`http://127.0.0.1:${address.port}/`], { timeout: 20_000, windowsHide: true }, () => {});
|
||||
chromeClosed = new Promise((resolve) => { chrome!.once("close", () => resolve()); });
|
||||
const chromeFailure = new Promise<never>((_, reject) => {
|
||||
chrome!.once("error", () => reject(new Error("Chrome failed to start")));
|
||||
chrome!.once("exit", () => reject(new Error("Chrome exited before TURN allocation completed")));
|
||||
});
|
||||
const result = await Promise.race([resultPromise, chromeFailure,
|
||||
new Promise<never>((_, reject) => { timeout = setTimeout(() => reject(new Error("Chrome did not complete the TURN allocation gate")), 15_000); })]);
|
||||
process.stdout.write(`${JSON.stringify({ schemaVersion: 1, ...result })}\n`);
|
||||
return result.status === "passed" ? 0 : 1;
|
||||
} finally {
|
||||
if (timeout) clearTimeout(timeout);
|
||||
if (chrome?.exitCode === null) chrome.kill();
|
||||
const serverClose = server.listening
|
||||
? new Promise<void>((resolve) => { server.close(() => resolve()); })
|
||||
: Promise.resolve();
|
||||
await Promise.allSettled([chromeClosed, serverClose]);
|
||||
await rm(profile, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) {
|
||||
main().then((code) => { process.exitCode = code; }).catch((error: Error) => {
|
||||
process.stderr.write(`${error.message}\n`); process.exitCode = 2;
|
||||
});
|
||||
}
|
||||
@@ -202,7 +202,7 @@ function parseStunUrlList(value: string | undefined): string[] {
|
||||
});
|
||||
}
|
||||
|
||||
function parseSelectedEdgeTurn(
|
||||
export function parseSelectedEdgeTurn(
|
||||
environment: NodeJS.ProcessEnv,
|
||||
): SelectedEdgeTurnConfig | undefined {
|
||||
const names = [
|
||||
|
||||
@@ -0,0 +1,48 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { parseGateEnvironment, parseGateResult } from "../scripts/turn-allocation-gate";
|
||||
|
||||
describe("TURN allocation gate", () => {
|
||||
it("parses credential input without truncating values containing equals", () => {
|
||||
expect(
|
||||
parseGateEnvironment(
|
||||
"SELECTED_EDGE_TURN_URLS=turn:relay.example:3478?transport=udp\n" +
|
||||
"SELECTED_EDGE_TURN_SHARED_SECRET=secret=value==\n",
|
||||
),
|
||||
).toEqual({
|
||||
SELECTED_EDGE_TURN_URLS: "turn:relay.example:3478?transport=udp",
|
||||
SELECTED_EDGE_TURN_SHARED_SECRET: "secret=value==",
|
||||
});
|
||||
});
|
||||
|
||||
it("returns only allowlisted allocation fields from Chrome output", () => {
|
||||
const secret = "must-not-leak";
|
||||
const result = parseGateResult(
|
||||
'{"status":"passed","relayCandidateCount":1,"protocols":["udp","udp"],' +
|
||||
`"errorCodeBuckets":[],"credential":"${secret}"}`,
|
||||
);
|
||||
|
||||
expect(result).toEqual({
|
||||
status: "passed",
|
||||
relayCandidateCount: 1,
|
||||
protocols: ["udp"],
|
||||
errorCodeBuckets: [],
|
||||
});
|
||||
expect(JSON.stringify(result)).not.toContain(secret);
|
||||
});
|
||||
|
||||
it("rejects pending or raw Chrome output with a fixed message", () => {
|
||||
expect(() => parseGateResult('"pending"')).toThrow(
|
||||
"Chrome returned an invalid TURN allocation result",
|
||||
);
|
||||
expect(() => parseGateResult("credential=must-not-leak")).toThrow(
|
||||
"Chrome returned an invalid TURN allocation result",
|
||||
);
|
||||
expect(() => parseGateResult(
|
||||
'{"status":"passed","relayCandidateCount":0,"protocols":[],"errorCodeBuckets":[]}',
|
||||
)).toThrow("Chrome returned an invalid TURN allocation result");
|
||||
expect(() => parseGateResult(
|
||||
'{"status":"passed","relayCandidateCount":1,"protocols":["tcp"],"errorCodeBuckets":[]}',
|
||||
)).toThrow("Chrome returned an invalid TURN allocation result");
|
||||
});
|
||||
});
|
||||
@@ -20,6 +20,7 @@
|
||||
"vitest.config.ts",
|
||||
"scripts/native-one-viewer-gate.ts",
|
||||
"scripts/native-one-viewer-ledger.ts",
|
||||
"scripts/turn-allocation-gate.ts",
|
||||
"tests"
|
||||
]
|
||||
}
|
||||
Reference in new issue
Block a user