fix: finish native client lifecycle and capture aspect
Accepted native Client lifecycle and Windows capture aspect phase. All Web/Server and three-platform Client checks passed.
This commit is contained in:
AAA authored and GitHub committed
1 parent
f95c420717
commit
9579a91168
181 files changed
+19370
-4239
No files matched your search
@@ -47,6 +47,8 @@ ordinary implementation and UI detail.
|
||||
audio boundary.
|
||||
- [ADR-0009](./adr/0009-optional-nat-prediction.md): optional connection-local
|
||||
NAT prediction.
|
||||
- [ADR-0011](./adr/0011-browser-assisted-native-fanout.md): Browser capture with
|
||||
Client encoded fanout.
|
||||
- [ADR-0010](./adr/0010-cross-platform-client-runtime.md): cross-platform Client
|
||||
runtime and self-contained package boundary.
|
||||
|
||||
|
||||
@@ -89,8 +89,16 @@ root. A fresh healthy candidate sender proof is required before make-before-brea
|
||||
commit. Success moves one branch toward even fanout; failure consumes that one-shot intent. No timer,
|
||||
continuous rebalance or general load score is introduced.
|
||||
|
||||
A failed Peer tuple is consumed for its exact parent and endpoint sessions plus
|
||||
endpoint-transition strength. SFU create, reuse, and replace are resource
|
||||
A Peer opportunity is fenced by its exact parent and endpoint sessions plus
|
||||
endpoint-transition strength. With NAT traversal disabled, one failed attempt
|
||||
consumes it. With traversal enabled, availability acquisition and direct
|
||||
continuation may create at most three candidate connections for that opportunity.
|
||||
Only an admitted candidate creation spends an attempt; duplicate failure and
|
||||
timeout messages cannot spend it again. A preparation rejection that cannot
|
||||
create a connection remains terminal. A new session or strictly stronger
|
||||
transition reopens the opportunity; worse transitions use only its remaining
|
||||
budget. Quality and root-convergence experiments retain their existing one-shot
|
||||
consumption rules. SFU create, reuse, and replace are resource
|
||||
actions for one logical SFU opportunity keyed by the child and Host sessions;
|
||||
the publication generation remains a physical fence, not retry authority.
|
||||
Controller-owned publication creation, replacement, or teardown cannot make the
|
||||
@@ -140,6 +148,18 @@ Peer candidates converge behind working media, one at a time and round-robin
|
||||
across SFU Viewers. New join or repair work preempts this background convergence.
|
||||
Exhausting direct candidates simply keeps the working SFU route.
|
||||
|
||||
When NAT acquisition exhausts one operation but retains candidate budget, the
|
||||
same serial scheduler starts the next operation after rollback and resource
|
||||
release. Each operation retains its full original deadline; neither a failed
|
||||
candidate nor a retry extends a running operation. Untried parents precede
|
||||
repeat attempts, and waiting Viewers with fewer started Peer attempts go first.
|
||||
The foreground SFU window is unchanged. SFU commitment carries the same
|
||||
remaining Peer budget into the existing round-robin direct continuations, not
|
||||
another retry ledger. No usable SFU route is interrupted by these attempts.
|
||||
This permits longer total acquisition while keeping every operation bounded and
|
||||
requiring no polling or independent retry timer. The prepare message reports
|
||||
the actual candidate's connection attempt, not inferred UI progress.
|
||||
|
||||
The five-second no-transport-progress window advances a silent Peer only while
|
||||
another bounded candidate or SFU fallback remains. It is a route scheduling
|
||||
boundary, not evidence that Browser ICE has reached a terminal state. The final
|
||||
|
||||
@@ -1,20 +1,21 @@
|
||||
# ADR-0009: Optional Connection-Local NAT Prediction
|
||||
|
||||
- Status: accepted as an optional deployment capability
|
||||
- Status: accepted as an optional Site and Client capability
|
||||
- Date: 2026-09-02
|
||||
|
||||
## Context
|
||||
|
||||
The self-hosted deployment has two auxiliary STUN-only listeners for controlled
|
||||
NAT measurements. A disposable namespace lab showed that a sequential,
|
||||
NAT measurements, while a self-contained Client needs public discovery without
|
||||
depending on that deployment. A disposable namespace lab showed that a sequential,
|
||||
endpoint-dependent mapping can sometimes be reached when the remote peer
|
||||
receives a small set of adjacent server-reflexive candidates. The result is
|
||||
network- and generation-specific; it is not a reliable NAT type or participant
|
||||
capability.
|
||||
|
||||
The existing deployment configuration, route policy, and ICE candidate owners
|
||||
are sufficient. The mechanism needs no NAT label, route-controller state, new
|
||||
media path, or third-party STUN dependency.
|
||||
The existing ICE configuration, route policy, and candidate owners are
|
||||
sufficient. Prediction needs no NAT classification or new media path; bounded
|
||||
connection acquisition reuses the route controller's existing opportunity ledger.
|
||||
|
||||
## Decision
|
||||
|
||||
@@ -23,35 +24,47 @@ media path, or third-party STUN dependency.
|
||||
policy. An enabled deployment exposes `NAT traversal`; the pre-share switch
|
||||
defaults on, remains Host-controllable, locks while sharing, and is not
|
||||
persisted beyond that share generation.
|
||||
2. An enabled deployment derives ports 3479 and 3480 from the first ordinary
|
||||
STUN authority on UDP 3478. When the share switch is on, every Browser P2P
|
||||
connection adds those same-host endpoints. This covers Host direct, Viewer
|
||||
upstream, and Viewer relay connections. SFU PeerConnections remain
|
||||
unchanged. No independent or third-party STUN endpoint is added.
|
||||
2. An enabled Site derives ports 3479 and 3480 from its first ordinary STUN
|
||||
authority on UDP 3478. Public-link Client mode instead supplies one ordinary
|
||||
public STUN destination and two fixed public survey destinations; pure LAN
|
||||
mode supplies none. The same share switch and ICE configuration cover every
|
||||
Browser or Native P2P edge. SFU PeerConnections remain unchanged.
|
||||
3. For one ICE gathering generation, the adapter observes only UDP `srflx`
|
||||
candidates whose Browser-reported STUN URL belongs to the self-hosted
|
||||
3478/3479/3480 survey set. If three or more distinct survey candidates for
|
||||
candidates belonging to the exact three-destination survey set. Browser
|
||||
candidates are matched by their reported STUN URL. Native Pion uses its
|
||||
`UniversalUDPMux` to perform the survey on the media socket and emits standard
|
||||
srflx-shaped observations marked with an `ns` foundation. Only those explicit
|
||||
survey observations feed prediction; port-mapped candidates remain ordinary
|
||||
ICE inputs. If three or more distinct survey candidates for
|
||||
one media section and public address form an arithmetic port sequence, the
|
||||
adapter appends at most eight bounded candidates outward from both sequence
|
||||
endpoints: four above the high endpoint and four below the low endpoint.
|
||||
Every ordinary candidate keeps normal Trickle ICE timing; no ordinary
|
||||
Every host and observed srflx candidate keeps normal Trickle ICE timing; no ordinary
|
||||
candidate is delayed, rejected, or replaced. A Browser that does not expose
|
||||
candidate source URLs keeps ordinary ICE without prediction. Rejection of a
|
||||
predicted remote candidate discards only that optional candidate; ordinary
|
||||
candidate errors retain their normal connection error semantics.
|
||||
4. Candidate observations remain in memory for that connection only. No NAT
|
||||
label, raw address, port, score, hard candidate skip, periodic probe, or
|
||||
route-controller branch is introduced. Sanitized diagnostics record only
|
||||
4. Candidate observations remain in memory for that connection only. They do
|
||||
not create persistent NAT labels, endpoint addresses, routing scores, hard
|
||||
candidate skips, or periodic probes. Sanitized diagnostics record only
|
||||
whether a signaled candidate was ordinary, predicted, or end-of-candidates,
|
||||
and whether the selected remote foundation was ordinary, predicted, or
|
||||
unknown.
|
||||
5. The same per-share gate also enables the three-attempt Peer acquisition
|
||||
budget in [ADR-0005](0005-automatic-hybrid-media-routing.md). Browser and
|
||||
Native candidates use the ordinary server-owned prepare/rollback lifecycle;
|
||||
no Viewer-owned ICE-restart loop is added. Local public-link and Hosted
|
||||
peer-only rooms can exhaust that budget without SFU, while mixed rooms use
|
||||
the remaining attempts behind a working SFU route. A new connection can
|
||||
provide another mapping opportunity, not guaranteed independent randomness:
|
||||
in particular, Native connections may retain the same shared UDP socket.
|
||||
|
||||
## Consequences
|
||||
|
||||
Positive:
|
||||
|
||||
- deployments that operate the required STUN listeners can offer the bounded
|
||||
mechanism while smaller deployments retain stock ICE;
|
||||
- Sites with their own listeners and self-contained public-link Clients can use
|
||||
the same bounded mechanism while pure Browser/LAN operation stays unchanged;
|
||||
- normal ICE remains immediately available, so an unavailable auxiliary STUN
|
||||
listener or unsuccessful prediction cannot delay direct or SFU fallback;
|
||||
- prediction is bounded to one connection generation and one small candidate
|
||||
@@ -63,24 +76,38 @@ Negative:
|
||||
|
||||
- the Browser may still reject synthetic candidates or the NAT may not have a
|
||||
matching mapping;
|
||||
- enabled P2P connections perform two extra self-hosted STUN transactions and
|
||||
may add connectivity checks for up to eight adjacent UDP ports; and
|
||||
- enabled P2P connections perform two extra STUN transactions and
|
||||
may add connectivity checks for up to eight adjacent UDP ports;
|
||||
- two endpoint-dependent NATs remain sensitive to candidate scheduling and
|
||||
intervening mappings, so room-wide coverage is not a success guarantee.
|
||||
intervening mappings, so room-wide coverage is not a success guarantee; and
|
||||
- enabled acquisition can occupy up to three ordinary operation windows when
|
||||
one parent remains, yielding between operations to other waiting Viewers.
|
||||
|
||||
Predicted connectivity checks originate from participant Browsers and target
|
||||
the other participant's bounded candidate window. The Screener server only
|
||||
receives Binding requests on its fixed 3478/3479/3480 listeners; enabling this
|
||||
capability does not make the cloud host scan external ports.
|
||||
Predicted connectivity checks originate from participant ICE agents and target
|
||||
the other participant's bounded candidate window. A Site server only receives
|
||||
Binding requests on its fixed listeners; it does not scan external ports.
|
||||
Public survey operators receive STUN metadata but never the media stream.
|
||||
|
||||
The controlled lab result (14/15 sequential-to-restricted runs with a bounded
|
||||
prediction set) is mechanism evidence, not a production success guarantee.
|
||||
Target-network prevalence, candidate direction, and long-run resource impact
|
||||
remain acceptance work.
|
||||
|
||||
A Windows Native Host preflight first exposed three different temporary local
|
||||
ports under Pion's ordinary srflx gatherer. Replacing that boundary with Pion's
|
||||
`UniversalUDPMux` produced an srflx observation whose related port exactly
|
||||
matched the one media listener. A subsequent public-link run delivered 35 H.264
|
||||
RTP packets to an independent Linux Pion Viewer over a selected direct
|
||||
host-to-srflx pair. This proves the shared-socket candidate carries DTLS-SRTP;
|
||||
the tested TUN network exposed only one distinct public mapping, so it does not
|
||||
prove a predicted candidate win.
|
||||
The current Native path maps selected Pion candidate foundations to the same
|
||||
anonymous `ordinary | predicted | unknown` evidence as Browser ICE.
|
||||
|
||||
## Primary Sources
|
||||
|
||||
- [RFC 8445: Interactive Connectivity Establishment](https://www.rfc-editor.org/rfc/rfc8445)
|
||||
- [RFC 4787: UDP NAT behavioral requirements](https://www.rfc-editor.org/rfc/rfc4787)
|
||||
- [RFC 5780: NAT Behavior Discovery Using STUN](https://www.rfc-editor.org/rfc/rfc5780)
|
||||
- [W3C WebRTC candidate model](https://www.w3.org/TR/webrtc/)
|
||||
- [Pion Universal UDP mux](https://github.com/pion/ice/blob/main/udp_mux_universal.go)
|
||||
@@ -1,7 +1,7 @@
|
||||
# ADR-0010: Cross-Platform Client Runtime
|
||||
|
||||
- Status: accepted Client/Local foundation and Windows native Host media boundary
|
||||
- Date: 2026-09-04
|
||||
- Status: accepted capability-provider architecture and native endpoint media boundary
|
||||
- Date: 2026-09-05
|
||||
|
||||
## Context
|
||||
|
||||
@@ -14,27 +14,37 @@ Existing Screener room, admission, signaling, and routing behavior already has
|
||||
one TypeScript implementation. Rewriting that behavior in the Helper would make
|
||||
Hosted and Local deployments diverge without improving the media path.
|
||||
|
||||
Room authority and local media capability are independent choices. A room may
|
||||
come from Local, a temporary public link, or a configured Site while each
|
||||
participant independently uses Browser media or available Client capability.
|
||||
Binding these choices into exclusive Client modes prevents mixed Browser/Native
|
||||
topologies and makes a saved Site unavailable while another room source runs.
|
||||
|
||||
## Decision
|
||||
|
||||
1. The TypeScript/Node core remains the sole owner of HTTP, room authority,
|
||||
admission, signaling, and routing in every deployment. A future local package
|
||||
runs that same server and the same built Browser assets with local
|
||||
configuration; Go does not reimplement the product core.
|
||||
2. The packaged product is Screener Client. Its Go process is the one
|
||||
cross-platform user entry and future native-media owner.
|
||||
Its current process exposes `/health` and one `/control` WebSocket on IPv4
|
||||
loopback within ports `39721` through `39730`.
|
||||
2. The packaged product is Screener Client. Its Go process is the cross-platform
|
||||
entry and native capability provider. It starts `/health` and one `/control`
|
||||
WebSocket on IPv4 loopback within ports `39721` through `39730` before room-
|
||||
source selection and retains them until the process exits.
|
||||
3. Discovery returns a per-process `instanceToken`. The WebSocket subprotocol
|
||||
echoes it so the Browser connects to the process it discovered. This value is
|
||||
public process identity, not authentication. Origin and Host validation plus
|
||||
the Browser's local-network permission own the current Browser boundary.
|
||||
4. Loopback v5 starts with a strict `hello` handshake. Health discovery reports
|
||||
only separately probed native capture booleans. An active control session may
|
||||
public process identity, not authentication. The listener accepts the current
|
||||
Local Host origin and the one user-saved Site origin. Origin and Host
|
||||
validation plus the Browser's local-network permission own this boundary.
|
||||
4. Loopback v8 starts with a strict `hello` handshake. Health discovery reports
|
||||
only separately probed native capture booleans. Native Viewer receive/NAT
|
||||
remains available when capture or hardware encode is absent. An active control session may
|
||||
list local screen/window choices, request bounded previews, and own one share's generation-fenced SDP/ICE
|
||||
edges, including at most one loopback media bridge outside route-copy
|
||||
capacity; it carries no room password, Host token, Viewer grant, or route
|
||||
policy. The Browser forwards current Site signaling and remains the
|
||||
participant.
|
||||
policy. An activated participant tab opens this connection lazily on its first
|
||||
native action, reuses it across successive media generations, and closes it
|
||||
with the page; a picker, share, or room source does not own the socket. The
|
||||
Browser forwards current signaling and remains the participant.
|
||||
5. A platform package contains the Go entry, a pinned Node runtime, and the same
|
||||
server/client build used by Hosted Screener. It may also carry one process-
|
||||
isolated capture binary and the pinned `cloudflared` sidecar. The Go entry
|
||||
@@ -42,7 +52,11 @@ Hosted and Local deployments diverge without improving the media path.
|
||||
metadata around that entry, not another long-running wrapper or UI.
|
||||
6. The system Browser remains the UI. Browser extensions, userscripts, Electron,
|
||||
Tauri, and resident services need new evidence before they can replace this
|
||||
smaller boundary.
|
||||
smaller boundary. A Client-opened Site stores a non-secret opt-in at that
|
||||
exact Browser origin, so later manually opened pages may discover the running
|
||||
Client. Pages without that opt-in do not probe localhost or request local-
|
||||
network permission. Clearing Site data simply requires opening it from the
|
||||
Client again.
|
||||
7. A self-contained Local deployment serves reachable LAN peers without a
|
||||
central Screener service. Its explicit `--link` mode starts one accountless
|
||||
Cloudflare Quick Tunnel for the same Node HTTP/WebSocket surface, injects the
|
||||
@@ -52,13 +66,24 @@ Hosted and Local deployments diverge without improving the media path.
|
||||
Browser. Cloudflare terminates this temporary control path; WebRTC media stays
|
||||
P2P and uses public STUN. The link ends with the Client and is not a persistent
|
||||
Site, SFU, or TURN fallback.
|
||||
8. Native media is selected for an entire Host share generation. One isolated
|
||||
8. Browser and Native are local media adapters beneath the same authenticated
|
||||
participant, connection identity, copy capacity, and committed route graph.
|
||||
Browser-to-Browser, Native-to-Browser, and Native-to-Native edges use the same
|
||||
WebRTC signaling contract. Native media currently covers the Host adapter;
|
||||
Viewer receive/relay follows the same boundary rather than adding a second
|
||||
participant or route protocol. A Native Viewer accepts only a negotiated
|
||||
H.264/VP8 and Opus offer, forwards encoded RTP into bounded local sources, and uses
|
||||
the existing Browser bridge for playback; compatible child edges reuse those
|
||||
sources. Unsupported media or a failed native bridge falls back to the
|
||||
existing Browser peer. One isolated
|
||||
platform capture feeds one encoded source and bounded independent Pion
|
||||
transports, with process-loopback audio for windows or system-loopback audio
|
||||
for screens sharing the same PeerConnection when available. Each Site or one-link share makes one bounded, best-effort PCP,
|
||||
UPnP, or NAT-PMP mapping for that same Pion UDP socket before its first edge
|
||||
gathers ICE; pure LAN Local mode does not. Absence or rejection leaves
|
||||
ordinary ICE/STUN unchanged. Native P2P quality uses Pion's transport-wide
|
||||
gathers ICE; pure LAN Local mode does not. Once ordinary STUN observes a
|
||||
public address, Native advertises the mapped port as one lower-priority,
|
||||
srflx-shaped candidate on that address. Absence or rejection leaves ordinary
|
||||
ICE/STUN unchanged. Native P2P quality uses Pion's transport-wide
|
||||
GCC estimate only after feedback and source-frame progress, comparing its
|
||||
target payload bitrate with the measured encoded payload supplied to that
|
||||
edge. The existing route evidence windows own persistence; no custom score,
|
||||
@@ -69,24 +94,44 @@ Hosted and Local deployments diverge without improving the media path.
|
||||
implement LiveKit or another representation policy. The Client discovers
|
||||
packaged capture capability at startup. Its Host page offers the Browser's
|
||||
standard picker and each exact Client-owned screen/window; the user must select one
|
||||
and the Client never guesses a target. Only a consumed Client-launch marker
|
||||
travels in the URL fragment. Exact target identity travels over loopback.
|
||||
9. The Client uses the system Browser as its only UI. A lightweight loopback
|
||||
launcher selects Local, one-link, or a saved Site before starting that
|
||||
composition, then navigates into the same application. Command-line mode
|
||||
and the Client never guesses a target. The same room quality settings select
|
||||
native capture size, frame rate, video/audio bitrate, and the platform
|
||||
encoder's quality-versus-speed hint. A live quality or source change prepares
|
||||
a replacement capture/encoder generation and swaps it behind the existing
|
||||
Pion source; room, route, and PeerConnections do not change. Windows uses
|
||||
Graphics Capture, Media Foundation, and WASAPI; macOS uses ScreenCaptureKit,
|
||||
VideoToolbox, and AudioToolbox; Linux uses the ScreenCast Portal, PipeWire,
|
||||
and an installed GStreamer hardware-H.264 element. These adapters end at the
|
||||
same bounded encoded-frame protocol and do not own WebRTC or product state.
|
||||
The shared Native encode is the normal path. For a persistently degraded
|
||||
Native sender edge, the existing quality operation may prepare an overlapping
|
||||
stock Browser sender from the stable local bridge; existing evidence alone
|
||||
decides commit or rollback, and a later operation may return to Native. No
|
||||
extra threshold, timer, score, route operation, or representation ladder is
|
||||
added. Only a consumed Client-launch marker travels in the URL fragment.
|
||||
Exact target identity travels over loopback.
|
||||
9. The Client uses the system Browser as its only UI. On every launch, the current
|
||||
lightweight control center offers Local, public link, and Site. The Site value
|
||||
is stored in Client configuration and remains one click on later launches;
|
||||
selecting Local or public link does not disable background RPC access for the
|
||||
saved Site. Command-line mode
|
||||
selectors remain automation inputs rather than the normal interface. An
|
||||
embedded shell requires a reproduced product failure and one comparative
|
||||
decision.
|
||||
10. Local mode is one explicit server composition: static current assets,
|
||||
memory-only rooms, peer-assisted media, no SQLite, no SFU, no NAT prediction,
|
||||
and localhost plus current LAN IPv4 origins. It uses no STUN by default;
|
||||
`--link` adds its exact temporary HTTPS origin and Cloudflare's public STUN
|
||||
to existing Browser media edges. It changes no Hosted shutdown or
|
||||
persistence behavior.
|
||||
11. One Client configuration owns the optional Site origin and a generated
|
||||
Local access password. The Client bootstraps its own Host page through a
|
||||
fragment that is consumed before authentication; friends use the existing
|
||||
room invitation grant. No Client-specific authorization system is added.
|
||||
memory-only rooms, peer-assisted media, no SQLite or SFU, and localhost plus
|
||||
current LAN IPv4 origins. It uses no public discovery by default. Public-link
|
||||
mode adds its temporary HTTPS origin, one ordinary public STUN destination,
|
||||
and two bounded public survey destinations. Site mode consumes that Site's
|
||||
configured STUN survey. Both feed the same connection-local prediction
|
||||
adapter; Native additionally owns its UDP socket and best-effort port mapping.
|
||||
11. One Client configuration owns the optional Site origin and an optional,
|
||||
user-chosen Local access password. A blank value leaves the Local site open;
|
||||
a value gates that site through the existing SiteAccess authority. The
|
||||
Client bootstraps a selected page through a fragment consumed before
|
||||
authentication; the page retains only the non-secret Client opt-in at its
|
||||
origin. Friends use the existing room invitation grant. No Client-specific
|
||||
room authorization system is added.
|
||||
12. Local authority shutdown first ends every in-memory room through the current
|
||||
`room-closed` path, then closes signaling and HTTP. The Go supervisor closes
|
||||
Node stdin, waits, and applies one bounded process timeout. It does not
|
||||
@@ -105,8 +150,9 @@ Hosted and Local deployments diverge without improving the media path.
|
||||
## Consequences
|
||||
|
||||
Hosted and Local operation share one product contract and one route model. The
|
||||
Go runtime acts as the Client's native-media owner when a Site or Local page
|
||||
discovers it, but it is not a second room product. It stays small until a
|
||||
Go runtime acts as an optional capability provider when an activated Site or
|
||||
Local page discovers it, but it is not a second room product. A mixed room does
|
||||
not expose endpoint implementation to routing policy. The runtime stays small until a
|
||||
proven native capability needs a protocol field. The local package may contain
|
||||
two internal processes while presenting one user entry; the supervisor, not a
|
||||
compatibility protocol, owns their lifetime.
|
||||
@@ -117,33 +163,41 @@ process-isolated hardware-H.264/Pion edges sharing one encoded source and
|
||||
decoded by Chrome, and bounded process cleanup. The native Host path uses the
|
||||
current Browser route; its explicit in-page window picker also passes source
|
||||
end, same-room reselection, and restored Viewer delivery. A remote Pion gate has
|
||||
received video over a direct
|
||||
`srflx`-to-`srflx` pair. A Windows Browser gate also receives process-loopback
|
||||
received 30+ H.264 RTP packets over a selected direct
|
||||
`srflx`-to-`srflx` pair; this proves packet delivery, not decoded Browser video.
|
||||
A Windows Browser gate also receives process-loopback
|
||||
Opus on both native edges. A separate remote gate proves that `--link` generates
|
||||
the ordinary public invitation and carries the unchanged Viewer page and
|
||||
WebSocket control path, then disappears when the Client exits. An isolated
|
||||
LiveKit gate also proves native capture through the loopback Browser bridge and
|
||||
the existing SFU publisher to 1280x720 Viewer playback with complete cleanup.
|
||||
Physical macOS capture, Linux native capture, and one-link Browser media remain
|
||||
separate gates.
|
||||
the existing SFU publisher, including a live 1080p-to-480p profile change and
|
||||
complete cleanup.
|
||||
A Browser-Host-to-Native-Viewer gate additionally proves that the Viewer claims
|
||||
the v8 Client control session and Chrome decodes the 1280x720 source; the encoded
|
||||
downstream edge has a separate H.264/Opus RTP integration gate.
|
||||
One-link Browser media and physical non-Windows capture remain separate gates.
|
||||
GitHub runners compile all three platform adapters. The macOS arm64 sidecar also
|
||||
creates a hardware-only VideoToolbox encoder and produces a constrained-baseline
|
||||
SPS/PPS/IDR; its ScreenCaptureKit video/audio permission, source lifecycle,
|
||||
static-screen recovery, and endurance still require a physical Mac. The Linux
|
||||
candidate probes and packages its Portal/PipeWire/GStreamer adapter, while a
|
||||
real desktop, hardware encoder, system audio, and lifecycle still require a
|
||||
physical Linux gate. Source previews remain best-effort on both platforms.
|
||||
|
||||
The macOS arm64 capture sidecar compiles on GitHub `macos-15` and its
|
||||
permission-free VideoToolbox self-test produces a constrained-baseline SPS/PPS/
|
||||
IDR. Real ScreenCaptureKit permission, source lifecycle, static-screen recovery,
|
||||
and endurance remain physical acceptance gates. Source previews are best-effort;
|
||||
the current macOS sidecar falls back to its source glyph until a physical
|
||||
ScreenCaptureKit preview gate justifies a platform-specific implementation.
|
||||
|
||||
The v5 loopback gate also proves that a real Chrome receiver produces Pion
|
||||
The v8 loopback gate also proves that a real Chrome receiver produces Pion
|
||||
transport feedback and that a non-unknown native sender-quality window reaches
|
||||
the existing route controller. Unknown feedback and stopped-source windows stay
|
||||
ineligible.
|
||||
|
||||
The local trust boundary is intentionally per-user: the instance token identifies
|
||||
the running Client but is not authentication against another local process.
|
||||
|
||||
A physical home-router gate created and removed a UPnP mapping for an ephemeral
|
||||
UDP listener. The integrated native Host gate still passed capture, two-edge
|
||||
delivery, source restart, and cleanup with mapping enabled. A selected mapped
|
||||
path across a pair that fails with STUN alone remains field evidence rather than
|
||||
an architectural claim.
|
||||
delivery, source restart, and cleanup with mapping enabled. The mapped port is
|
||||
now advertised on the observed public address; a selected mapped path across a
|
||||
pair that fails with STUN alone remains field evidence rather than an
|
||||
architectural claim.
|
||||
|
||||
## Primary Sources
|
||||
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
# ADR-0011: Browser-Assisted Native Encoded Fanout
|
||||
|
||||
- Status: accepted
|
||||
- Date: 2026-09-05
|
||||
|
||||
## Context
|
||||
|
||||
The product media goal is one encoded source per capable distribution layer.
|
||||
Pure Browser WebRTC does not expose a way to inject a received encoded frame
|
||||
into another `RTCPeerConnection` sender. `RTCRtpScriptTransform` can observe or
|
||||
modify an existing sender or receiver pipeline, but it does not provide an RTP
|
||||
transport fanout API. Browser simulcast and Dynacast remain appropriate for a
|
||||
single Browser-to-SFU publication; they do not make independent P2P senders
|
||||
share one encoder.
|
||||
|
||||
The repository already has the required Native boundary: a Pion receiver can
|
||||
accept H.264/Opus RTP and its encoded source can feed bounded downstream Pion
|
||||
edges without decoding or re-encoding. The system Browser remains the product
|
||||
UI and capture owner.
|
||||
|
||||
## Decision
|
||||
|
||||
1. A Client-launched Browser Host whose current codec is H.264 and whose
|
||||
topology optimization is enabled uses one local Browser-to-Native ingress.
|
||||
The existing `HostPeer` owns its sender, clone, quality settings and pause;
|
||||
the existing Native receiver owns encoded fanout. VP8 and disabled quality
|
||||
convergence retain ordinary Browser senders.
|
||||
2. The ingress is generation-fenced beneath the existing room and route
|
||||
controller. It adds no participant, route operation, quality score, or
|
||||
representation policy. Steady P2P output uses one Browser encoding; a LiveKit
|
||||
publication still owns its separate framework-managed encodings.
|
||||
3. Native downstream edges reuse the existing `NativeSenderPeer` and
|
||||
`sourceConnectionId` path. A quality operation may still use the existing
|
||||
Browser sender candidate for one degraded edge; that candidate is an
|
||||
exception when sharing an encoding cannot sustain that path. Savings never
|
||||
justify worse delivery or disabling native Browser quality adaptation.
|
||||
4. If the Client is absent or the local ingress fails, the Host keeps the
|
||||
Browser capture and recreates affected senders on their current assigned
|
||||
routes. Capture and room authority survive loss of the optional fanout.
|
||||
Ordinary Browser pages retain their existing media path.
|
||||
5. Native capture, Browser capture through the local ingress, Native Viewer
|
||||
relay, and LiveKit SFU publication remain separate media adapters. No
|
||||
application RTP protocol, WebCodecs object protocol, SVC ladder, codec
|
||||
parser, or custom congestion controller is added.
|
||||
|
||||
## Consequences
|
||||
|
||||
- A Client-launched Browser Host can use one Browser encode plus Native encoded
|
||||
fanout while preserving the existing UI and room behavior.
|
||||
- A pure Browser relay remains framework-owned and may encode per child because
|
||||
the standard Browser API has no cross-connection encoded-frame injection
|
||||
surface.
|
||||
- A Browser-to-Native loopback edge adds local WebRTC overhead, but it replaces
|
||||
repeated remote Browser encoders. Pion owns downstream ICE, DTLS and RTCP;
|
||||
its existing GCC observations feed route convergence, not a new layer selector.
|
||||
- A Browser source switch with unchanged media kinds reuses the local sender.
|
||||
A change in audio presence prepares a new ingress before replacing its Native
|
||||
children; failure retains Browser capture through ordinary sender recovery.
|
||||
- A Windows/Chrome two-child gate proves one local encode, 1080p at about 30 fps
|
||||
on both Viewers, live quality changes, pause/resume, source audio changes, and
|
||||
Client-exit recovery. Weak-path automatic adaptation and its cost remain
|
||||
physical acceptance, not a consequence of the clean-path result.
|
||||
|
||||
## References
|
||||
|
||||
- [Advanced peer distribution research](../research/advanced-peer-distribution.md)
|
||||
- [WebRTC Encoded Transform](https://www.w3.org/TR/webrtc-encoded-transform/)
|
||||
- [WebRTC SVC](https://www.w3.org/TR/webrtc-svc/)
|
||||
- [Pion encoded broadcast](https://github.com/pion/webrtc/blob/main/examples/broadcast/main.go)
|
||||
- [Pion RTP forwarding and padding](https://github.com/pion/webrtc/blob/main/track_local_static.go)
|
||||
@@ -63,6 +63,12 @@ is one of `windows-amd64`, `linux-amd64`, or `darwin-arm64`; every supplied
|
||||
runtime must match it. It does not create an installer, auto-updater, release
|
||||
tag, or compatibility bundle.
|
||||
|
||||
The platform package also carries its native presentation metadata: Windows
|
||||
embeds the icon in the Go executable, Linux emits a freedesktop desktop entry
|
||||
under `share/`, and macOS emits a thin `.app` launcher with an ICNS resource.
|
||||
The latter two are packaging metadata only and do not duplicate the Client or
|
||||
Node runtime.
|
||||
|
||||
CI and local release-candidate builds use the same wrapper on the target's
|
||||
native operating system:
|
||||
|
||||
|
||||
+6
-6
@@ -4,7 +4,7 @@
|
||||
|
||||
仓库是跨 session、设备和 agent 的持久事实源。会影响后续工作的结论必须先进入其权威文档；聊天、agent 摘要、旧分支和临时计划都不能替代仓库真相。
|
||||
|
||||
常驻上下文只保留每次工作都需要的边界和导航。细节按需读取，完成历史交给 Git。任务范围、最小正确结果和 current-result 交付遵循 `AGENTS.md` 摘要并完整读取仓库内的 [`stop-that-shit` skill](../.agents/skills/stop-that-shit/SKILL.md)；本文件只维护仓库特有的事实治理规则。
|
||||
常驻上下文只保留每次工作都需要的边界和导航。细节按需读取，完成历史交给 Git。任务范围和交付遵循 `AGENTS.md`，最小正确结果遵循仓库内的 [`ponytail` skill](../.agents/skills/ponytail/SKILL.md)；本文件只维护仓库特有的事实治理规则。
|
||||
|
||||
## 事实分层
|
||||
|
||||
@@ -25,13 +25,13 @@
|
||||
|
||||
## 顶级指令卫生
|
||||
|
||||
[官方调研](./research/agent-context-governance.md)的共同结论是：顶层指令应是短而稳定的地图，流程和专业细节按需加载，并定期删除冲突或陈旧规则。检查器把官方边界分成推荐线和硬顶：Codex 项目指令链在默认 32 KiB 上限的 80% 提示、到上限失败；Claude 的有效启动指令在官方建议的 200 行提示，并容许到 250 行的项目缓冲；Hermes 在动态下限 20,000 字符的 80% 提示、到下限失败。`CLAUDE.md` 导入的 `AGENTS.md` 计入有效行数；每次任务必读的 repo STS skill 使用同一保守包络。
|
||||
[官方调研](./research/agent-context-governance.md)的共同结论是：顶层指令应是短而稳定的地图，流程和专业细节按需加载，并定期删除冲突或陈旧规则。检查器把官方边界分成推荐线和硬顶：Codex 项目指令链在默认 32 KiB 上限的 80% 提示、到上限失败；Claude 的有效启动指令在官方建议的 200 行提示，并容许到 250 行的项目缓冲；Hermes 在动态下限 20,000 字符的 80% 提示、到下限失败。`CLAUDE.md` 导入的 `AGENTS.md` 计入有效行数；必读的 repo Ponytail skill 使用同一保守包络。
|
||||
|
||||
推荐线只触发园艺提醒，不是压缩目标。硬顶失败时应把目录专属规则迁到嵌套或 path-scoped context，把流程和参考资料迁到按需 skill/文档，简化重复措辞，并删除已完成历史或陈旧结论；不要为了过闸损失仍有消费者的约束。模块特有规则只有在对应代码存在且每次读取确有必要时，才放到最近目录的 `AGENTS.md`；流程说明进入 `CONTRIBUTING.md`，当前工作进入 `docs/todo.md`，证据与理由进入其 owner。
|
||||
|
||||
## 真相更新顺序
|
||||
|
||||
1. 按 [`stop-that-shit`](../.agents/skills/stop-that-shit/SKILL.md) 的 authority 和 scope 规则，对照完整产品模型、当前代码、测试和证据确定目标；语义未定时只冻结依赖项。
|
||||
1. 按 [`AGENTS.md`](../AGENTS.md) 的 authority 和 scope 规则，对照完整产品模型、当前代码、测试和证据确定目标；语义未定时只冻结依赖项。
|
||||
2. 就地更新受影响的 product、ADR 或 research owner；仅在快照确实变化时同步 memory/status，未决执行项进入 TODO。
|
||||
3. 检查各 current owner 一致并形成 Git-tracked truth checkpoint，再开始依赖该结论的实现或派发。
|
||||
|
||||
@@ -39,7 +39,7 @@
|
||||
|
||||
## 机制减负审查
|
||||
|
||||
机制减负遵循 [`stop-that-shit`](../.agents/skills/stop-that-shit/SKILL.md) 的 scope 和 smallest-correct-result 规则。本仓库的审查先只读清点机制服务的用户行为，量化移除后的真实退让，并与代码、配置、迁移、测试、运行时状态、部署和故障面的整体成本比较。文档也是需要同步和审查的机制：长期真相只保留产品合同、非显然不变量、算法/模型、关键取舍和外部证据，普通 UI 与实现细节由代码、测试和 PR 持有。若 owner 接受有界退让，先更新产品合同再删除失去必要性的完整表面；否则保留合同并寻找更小实现。审查不以行数为目标，也不授权无边界重构。
|
||||
机制减负遵循 [`ponytail`](../.agents/skills/ponytail/SKILL.md) 的最小正确结果原则，范围由 `AGENTS.md` 约束。本仓库的审查先只读清点机制服务的用户行为，量化移除后的真实退让，并与代码、配置、迁移、测试、运行时状态、部署和故障面的整体成本比较。文档也是需要同步和审查的机制：长期真相只保留产品合同、非显然不变量、算法/模型、关键取舍和外部证据，普通 UI 与实现细节由代码、测试和 PR 持有。若 owner 接受有界退让，先更新产品合同再删除失去必要性的完整表面；否则保留合同并寻找更小实现。审查不以行数为目标，也不授权无边界重构。
|
||||
|
||||
TURN 的减负证据由 [ADR-0005](./adr/0005-automatic-hybrid-media-routing.md) 和相关 transport research 持有；轻量/SQLite 房间取舍由 [ADR-0002](./adr/0002-memory-resident-protected-rooms.md) 持有。这些案例只证明应按当前消费者重审完整成本，不能被推广为永远删除或永远保留某类机制的规则。
|
||||
|
||||
@@ -47,7 +47,7 @@ TURN 的减负证据由 [ADR-0005](./adr/0005-automatic-hybrid-media-routing.md)
|
||||
|
||||
在需求或优先级被接受、设计或研究结论改变、实质阶段切换、分支/PR/agent 交接、可能 compaction 或长暂停前更新事实 owner。不要为每个函数、工具调用或未形成结论的探索建立检查点。
|
||||
|
||||
检查点按 [`stop-that-shit`](../.agents/skills/stop-that-shit/SKILL.md) 的 current-result 规则，只保存以后仍需要的结论、当前状态、下一步和真实阻塞；不保存原始对话或工具输出、完整日志、临时路径、可廉价重查的事实和过程性 TODO 流水账。
|
||||
检查点按 [`AGENTS.md`](../AGENTS.md) 的 durable-truth 规则，只保存以后仍需要的结论、当前状态、下一步和真实阻塞；不保存原始对话或工具输出、完整日志、临时路径、可廉价重查的事实和过程性 TODO 流水账。
|
||||
|
||||
恢复上下文时依次读取 `AGENTS.md`、project memory、status、TODO、相关 product/ADR/research，然后检查当前分支、`git status`、近期 commit 和 PR。摘要与仓库冲突时，以当前代码、测试和权威文档为准。
|
||||
|
||||
@@ -63,6 +63,6 @@ TURN 的减负证据由 [ADR-0005](./adr/0005-automatic-hybrid-media-routing.md)
|
||||
|
||||
- 迭代中运行覆盖当前改动的最小定向检查；完整套件、真实浏览器矩阵、网络整形、跨设备、长时稳定性和部署检查在相关候选的验收边界批量运行。
|
||||
- 文档改动默认运行本地 Markdown 链接、whitespace、diff 和 repository-hygiene 检查；不因无关文档变化重跑媒体基准。
|
||||
- 断链、缺失必需 owner 和格式损坏是 hard failure。每次注入的有效 `AGENTS.md`/`CLAUDE.md` 上下文与每次必读的 repo STS skill 使用“推荐线 warning、硬顶 failure”，避免把建议值误当死线，也避免超过工具加载边界。Product、ADR、research、operations 等按需文档的行数/字节阈值始终只是 warning：它提示检查是否应拆分二级 owner、删除历史或移出过程信息，不要求为通过门禁压缩合理正文，也不是交付质量指标。
|
||||
- 断链、缺失必需 owner 和格式损坏是 hard failure。每次注入的有效 `AGENTS.md`/`CLAUDE.md` 上下文与必读的 repo Ponytail skill 使用“推荐线 warning、硬顶 failure”，避免把建议值误当死线，也避免超过工具加载边界。Product、ADR、research、operations 等按需文档的行数/字节阈值始终只是 warning：它提示检查是否应拆分二级 owner、删除历史或移出过程信息，不要求为通过门禁压缩合理正文，也不是交付质量指标。
|
||||
- 昂贵证据记录 commit、环境、结果和适用边界。只有相关路径、协议、配置、依赖、环境或门槛改变，或旧证据失败/含糊时才重跑。
|
||||
- `.githooks/pre-commit`、`scripts/check-project-state.sh` 和 PowerShell 等价入口共享 `scripts/required-project-paths.txt`。Hook 只做快速、确定、可复现的检查；工程判断留给评审和测试。
|
||||
@@ -1,6 +1,6 @@
|
||||
# Capture, Audio, And Media Quality
|
||||
|
||||
This file owns the current Browser media contract. Detailed measurements and
|
||||
This file owns the current Browser and Client media contract. Detailed measurements and
|
||||
platform limits live in [realtime quality research](../research/realtime-quality-adaptation.md)
|
||||
and [screen-audio research](../research/browser-screen-audio-quality.md).
|
||||
[ADR-0007](../adr/0007-path-isolated-representation-quality.md) owns the SFU
|
||||
@@ -12,21 +12,41 @@ adaptation decision.
|
||||
stop, synchronously pause/resume audio and video, or switch source.
|
||||
- A Client-launched Host explicitly chooses either that Browser capture path or
|
||||
one native screen/window enumerated by the packaged platform capture boundary. The
|
||||
latter uses one available hardware H.264 path and never infers a target from
|
||||
latter uses one supported native codec path and never infers a target from
|
||||
a title. An ordinary Web Host does not probe localhost.
|
||||
- Windows native capture follows an explicitly stretched active display path
|
||||
for an entire display or a window covering that display, when the captured
|
||||
frame matches its desktop source dimensions. Other frames retain their own
|
||||
aspect ratio. This changes only the encoded presentation, never the game or
|
||||
display settings; vendor-private scaling is not inferred.
|
||||
- Share and source-switch requests ask the Browser for available audio by
|
||||
default. Missing audio is reported clearly but does not block video-only
|
||||
sharing. Native screen capture can include system playback audio and native
|
||||
window capture can include selected-process audio when the platform exposes it.
|
||||
- Authoritative pause disables the current source and sender-owned tracks while
|
||||
retaining the room and established routes. Black frames, track mute, or
|
||||
network failure are not interpreted as a user pause.
|
||||
- Authoritative Browser pause disables the current source and sender-owned tracks
|
||||
while retaining the room and established routes. Native pause keeps capture
|
||||
alive but stops session output through the same owner. Black frames, track
|
||||
mute, or network failure are not interpreted as a user pause.
|
||||
- The Host preview displays the capture stream directly and creates no Viewer or
|
||||
media route. Hiding the page may pause only that local video element; it must
|
||||
not intentionally stop capture, encoding, or upload.
|
||||
- Live quality changes update the current capture and Host sender-track
|
||||
constraints plus sender ceilings without reopening source selection or
|
||||
replacing a healthy route.
|
||||
- Native quality changes use the same room settings. The Client prepares a new
|
||||
platform capture/encoder generation, then replaces the old generation behind
|
||||
the same encoded source and PeerConnections; audio-only changes update the
|
||||
current Opus encoder directly. Platform quality preference uses the hardware
|
||||
encoder's standard quality-versus-speed hint, while Pion/WebRTC still own
|
||||
transport estimation and route evidence.
|
||||
Windows VP8 uses libvpx's realtime mode; it does not claim the same hardware
|
||||
quality-versus-speed control or per-edge Browser adaptation.
|
||||
- With Client available, Browser H.264 capture can use one local sender and the
|
||||
existing Native encoded fanout while topology optimization is enabled. The
|
||||
Browser still owns preview, pause, capture settings, and source selection.
|
||||
Loss of that optional ingress retains capture and rebuilds the assigned
|
||||
Browser edges. [ADR-0011](../adr/0011-browser-assisted-native-fanout.md) owns
|
||||
this composition; pure Browser and VP8 sharing retain their normal senders.
|
||||
|
||||
## Video Profiles
|
||||
|
||||
@@ -70,6 +90,15 @@ The Browser chooses the concrete encoder implementation. A reported H.264 codec
|
||||
does not by itself prove hardware acceleration, and Web content cannot select a
|
||||
specific MFT, NVENC, AMF, or QSV implementation.
|
||||
|
||||
Windows native capture uses the same VP8/Auto/H264 controls. H264 selects the
|
||||
hardware path and VP8 the bundled libvpx encoder. Auto measures encoding work
|
||||
for synthetic NV12 frames at the selected dimensions and frame rate, with a
|
||||
bounded warmup and sample. If H264 sustains the target it is selected; otherwise
|
||||
VP8 is measured within the remaining four-second budget. This is a throughput
|
||||
check, not a perceptual-quality score or a promise under future GPU load.
|
||||
The returned actual codec owns the shared source, preview, and relay; live
|
||||
quality/source changes retain it. Other native platform encoders remain H264.
|
||||
|
||||
## Framework-Owned Adaptation
|
||||
|
||||
Each direct, relay, or SFU video sender owns one clone of its capture or received
|
||||
@@ -95,9 +124,16 @@ capture therefore retains source resolution and frame-rate ownership across its
|
||||
loopback Browser bridge; the existing Browser SFU publisher applies only sender
|
||||
parameters and LiveKit's representation policy to that remote source.
|
||||
|
||||
When a live native sender remains persistently limited after capacity recovers,
|
||||
the route controller may rebuild that exact edge with a fresh connection and
|
||||
clone; media adaptation still stays entirely framework-owned.
|
||||
Native sender edges normally reuse one encoded source. A Native Viewer forwards
|
||||
compatible H.264/VP8 and Opus payload without decoding or re-encoding it; each outbound
|
||||
PeerConnection owns its RTP identity and transport feedback. When an edge is
|
||||
persistently degraded, the existing quality operation may prepare a Browser
|
||||
WebRTC sender from the stable local bridge as that edge's candidate. The old
|
||||
edge stays live until the Viewer proves the candidate is better; a failed
|
||||
candidate rolls back, and a later operation may return the edge to the shared
|
||||
native source. This gives one difficult path stock per-sender adaptation without
|
||||
lowering the shared representation for healthy paths. It adds no quality score,
|
||||
timer, or room-wide media setting.
|
||||
|
||||
Screener does not maintain an application bitrate/resolution ladder, scene
|
||||
detector, periodic quality controller, manual SFU layer selector, or
|
||||
|
||||
@@ -19,7 +19,8 @@ desktop and mobile viewports.
|
||||
|
||||
Screener Client opens this same application in the system Browser. Its small
|
||||
startup surface selects Local, temporary public invitation, or a configured
|
||||
Site before entering the Host workspace. Starting a share there offers the
|
||||
Site before entering the Host workspace. Viewer links opened at that activated
|
||||
origin may use the same running Client without changing the Viewer UI. Starting a share offers the
|
||||
Browser picker and any exact Client-owned native windows. It never chooses a
|
||||
window automatically. A reproduced Browser-window, capture, or background
|
||||
failure is required before introducing an embedded Web runtime.
|
||||
@@ -40,9 +41,15 @@ and sequences, and literal scene objects for people, rooms, and media. Glyphs do
|
||||
not form sentences. Universal digits, transport symbols, URLs, and measured
|
||||
values remain literal when users need the data.
|
||||
|
||||
Motion explains entry, transition, progress, and control feedback. Stable state
|
||||
stays still; ambient animation and motion added only for ornament are outside
|
||||
the language. All meaning remains available with reduced motion. Functional
|
||||
Control hints and state comics use the same television for shared or watched
|
||||
media. Browser chrome identifies the application UI; a window title bar or a
|
||||
display stand identifies the capture target. A server in a media path means
|
||||
actual SFU relay, not merely opening a configured Site.
|
||||
|
||||
Motion explains entry, transition, progress, and control feedback. Operational
|
||||
status stays still; ambient motion is limited to the Client brand mark's
|
||||
occasional idle wink.
|
||||
All meaning remains available with reduced motion. Functional
|
||||
controls remain native buttons or inputs with localized accessible names, and
|
||||
hover, keyboard focus, and touch receive equivalent guidance.
|
||||
|
||||
@@ -123,10 +130,10 @@ that owns that PeerConnection and are never uploaded, persisted, or used for
|
||||
identity or route selection. The product does not offer a diagnostic-download
|
||||
button or a raw stats dump.
|
||||
|
||||
Current Browser scope includes Web Host, Web Viewer, and Browser relay. Desktop
|
||||
and mobile Browsers are Viewer targets; mobile Web capture, reliable background
|
||||
relay, native Client audio/SFU, and non-Windows capture remain later platform
|
||||
work.
|
||||
Current scope includes Web Host, Web Viewer, Browser relay, and optional Native
|
||||
Host/Viewer media beneath those same pages. Desktop and mobile Browsers are
|
||||
Viewer targets; mobile Web capture, reliable background relay, Client-owned SFU,
|
||||
and physical non-Windows native acceptance remain later platform work.
|
||||
|
||||
## Primary References
|
||||
|
||||
|
||||
@@ -74,13 +74,15 @@ code-only attempts and does not silently revoke invitations.
|
||||
room authority across application restart.
|
||||
- **Client Local mode:** the packaged Client composes the same memory RoomStore
|
||||
and ends every room when its local authority exits. It persists only its Site
|
||||
choice and generated Local site-access password, not rooms or media state.
|
||||
choice and optional Local site-access password, not rooms or media state. A
|
||||
blank password leaves that local site open.
|
||||
|
||||
The Client opens its own localhost Host page with that password in a fragment.
|
||||
The page removes the fragment and uses the existing SiteAccess endpoint; this
|
||||
does not create a fourth admission authority. LAN and one-link Internet Viewers
|
||||
use the same room-scoped invitation grant; its fragment never enters the public
|
||||
tunnel request, and the Host's Local RoomStore remains the only authority.
|
||||
The Client opens its own localhost Host page with the configured password, when
|
||||
present, in a fragment. The page removes the fragment and uses the existing
|
||||
SiteAccess endpoint; this does not create a fourth admission authority. LAN and
|
||||
one-link Internet Viewers use the same room-scoped invitation grant; its
|
||||
fragment never enters the public tunnel request, and the Host's Local RoomStore
|
||||
remains the only authority.
|
||||
Selecting a Site moves room authority wholly to that Site rather than
|
||||
synchronizing two stores.
|
||||
|
||||
|
||||
@@ -32,11 +32,16 @@ share generation:
|
||||
- native-edge topology convergence is enabled by default. The Host may disable
|
||||
it before sharing; when disabled, quality evidence stays diagnostic and
|
||||
availability routing is unchanged;
|
||||
- NAT traversal is absent unless the deployment enables its self-hosted
|
||||
auxiliary STUN capability. When available, its Host switch defaults on and
|
||||
augments every Browser P2P edge in that share; the Host may disable it before
|
||||
sharing. It is not a participant capability, route score, or SFU preference.
|
||||
Candidate generation and selected-pair diagnostics retain only
|
||||
- NAT traversal requires an exact three-destination STUN survey: a Site may use
|
||||
self-hosted 3478/3479/3480, Public Link uses its bounded public survey, and
|
||||
pure LAN supplies none. Its default-on Host switch augments Browser and Native
|
||||
P2P edges; Native also uses its media socket and best-effort gateway mapping.
|
||||
Availability and background direct acquisition share a bounded budget of
|
||||
three actual connection attempts per eligible parent/session opportunity.
|
||||
Retries use the same serial controller and normal operation deadlines;
|
||||
waiting Viewers and untried parents receive their opportunities first.
|
||||
It is not a participant capability, route score, or SFU preference. Candidate
|
||||
generation and selected-pair diagnostics retain only
|
||||
`ordinary | predicted | unknown` provenance.
|
||||
[ADR-0009](../adr/0009-optional-nat-prediction.md) owns the bounded behavior
|
||||
and evidence boundary.
|
||||
@@ -90,6 +95,9 @@ not abandon its only possible route at the foreground boundary.
|
||||
The pending Viewer transport reports an actual Browser failure but installs no
|
||||
shorter initial or disconnected-state deadline. Viewer-owned reconnect timing
|
||||
begins only after that exact candidate commits as the active route.
|
||||
When NAT traversal is enabled, another operation may use the remaining
|
||||
connection-attempt budget after rollback. The waiting display uses the actual
|
||||
server-issued attempt ordinal; it never counts time as an attempted connection.
|
||||
|
||||
When a newly committed Host-root Viewer exposes unused downstream capacity while
|
||||
another Host root has at least two direct children, the same background operation may
|
||||
@@ -163,17 +171,16 @@ encrypted hop-by-hop with DTLS-SRTP but terminates at the SFU; the product does
|
||||
not claim operator-blind media without a separately accepted application E2EE
|
||||
design. All-UDP-blocked networks currently end in bounded failure.
|
||||
|
||||
Client Local mode runs this same graph with peer-assisted media enabled and no
|
||||
SFU or NAT prediction. It uses no STUN by default. Explicit one-link mode carries
|
||||
only HTTP/WebSocket control through a temporary public tunnel and adds public
|
||||
STUN to the same ordinary Browser media edges. Reachable peers may form the
|
||||
ordinary Browser relay tree; an unreachable media path fails without adding a
|
||||
route type, score, or fallback. Selecting a Site uses that deployment's current
|
||||
transport configuration instead.
|
||||
Client Local mode runs this graph without SFU, NAT prediction, or default STUN.
|
||||
One-link mode tunnels only HTTP/WebSocket control and adds public STUN to the
|
||||
same Browser/Native P2P edges. Reachable peers form one mixed relay tree; an
|
||||
unreachable path adds no route type or score. Site mode uses Site transport.
|
||||
|
||||
A native Host reserves one additional loopback media edge for its system-
|
||||
Browser preview and existing Browser SFU publisher. That local edge neither
|
||||
consumes endpoint route-copy capacity nor starts gateway port mapping. Direct
|
||||
children still use the native shared encode; if the controller assigns SFU,
|
||||
the ordinary SFU publication consumes its existing route copy and retains the
|
||||
same LiveKit representation and recovery policy as Browser capture.
|
||||
A native Host reserves one loopback edge for Browser preview and the existing
|
||||
SFU publisher; it consumes no route-copy capacity or gateway mapping. Direct
|
||||
children share the native encode, while assigned SFU keeps the Browser LiveKit
|
||||
representation and recovery policy.
|
||||
|
||||
A native Viewer terminates only its P2P upstream, reserves one Browser playback
|
||||
edge, and reuses compatible H.264/Opus for bounded children. Browser keeps SFU,
|
||||
route revisions, frame proof, and fallback.
|
||||
+24
-12
@@ -1,6 +1,6 @@
|
||||
# Project Memory
|
||||
|
||||
Last updated: 2026-09-04
|
||||
Last updated: 2026-09-05
|
||||
|
||||
Screener is private, low-latency game screen sharing for one Host and up to 20
|
||||
authenticated friends. The current product surface is Web Host, Web Viewer, and
|
||||
@@ -8,18 +8,30 @@ Browser relay; desktop and mobile Browsers are Viewer targets. It is not a
|
||||
public broadcast service. Source includes a cross-platform Client that reuses
|
||||
the TypeScript product core in a self-contained Local deployment or opens one
|
||||
configured Site through the system Browser. Its Go entry presents Local,
|
||||
temporary public-link, and Site startup choices, supervises Local Node, and owns
|
||||
the explicit native Host media path selected in the same Host UI. Local remains
|
||||
temporary public-link, and saved-Site choices on every launch, starts one process-
|
||||
level loopback capability service, supervises Local Node when selected, and owns
|
||||
the explicit native media path chosen in the same Web UI. Room source and media
|
||||
implementation are independent: a topology may mix Browser and Native peers
|
||||
without changing participant, signaling, capacity, or route identity. Native
|
||||
Host and H.264/VP8 Viewer receive/relay adapters share the same Pion media edge;
|
||||
unsupported Viewer media falls back to Browser. A Client-assisted Browser Host
|
||||
can feed that fanout through one local H.264 sender while quality convergence is
|
||||
enabled; losing the Client preserves Browser capture. Local remains
|
||||
a serverless-LAN mode by default; explicit one-link mode exposes that same
|
||||
authority through a session-scoped public control tunnel while media remains
|
||||
P2P. Windows capture,
|
||||
hardware H.264, direct cross-NAT video, Windows process/system audio, and best-effort
|
||||
mapping of the native edge's sole UDP socket are implemented through the same
|
||||
media boundary. Native P2P edges feed Pion GCC's categorical payload-capacity
|
||||
result into the existing route evidence windows. One reserved loopback edge
|
||||
gives the system Browser a preview and lets native Host media reuse the existing
|
||||
LiveKit publisher when SFU fallback is assigned; other platform capture and
|
||||
broader package acceptance remain later gates.
|
||||
P2P. Minimal Windows, macOS, and Linux adapters terminate at one encoded-frame
|
||||
boundary; only Windows has completed physical native-media acceptance, while
|
||||
the other two compile and package. Native capture consumes the same room quality
|
||||
settings and applies live quality or source changes behind its stable encoded
|
||||
source and connections. Native P2P edges feed Pion GCC's categorical payload-
|
||||
capacity result into the existing route evidence windows. Healthy Native sender
|
||||
edges share one hardware encode; the existing quality operation may test one
|
||||
stock Browser sender for a persistently degraded edge without lowering the
|
||||
shared source. One reserved loopback edge gives the system Browser a preview and
|
||||
lets native Host media reuse the existing LiveKit publisher when SFU fallback is
|
||||
assigned; broader package acceptance remains a later gate.
|
||||
The Local launcher keeps an optional user-chosen site-access password; leaving it
|
||||
blank keeps the self-contained site open.
|
||||
|
||||
## Product Map
|
||||
|
||||
@@ -72,7 +84,7 @@ implementation and routine UI detail.
|
||||
|
||||
## Current Snapshot
|
||||
|
||||
Canonical source uses the strict `screener-v19` Browser/server contract with
|
||||
Canonical source uses the strict `screener-v20` Browser/server contract with
|
||||
committed media readiness, bounded candidate-relative progress and one-shot
|
||||
Host-root convergence. It retains optional SQLite room authority and the
|
||||
H.264/VP8 sender gate. [Status](./status.md) owns the compact current execution
|
||||
|
||||
@@ -38,7 +38,7 @@ Screener therefore warns at the published recommendation or 80% of a loader
|
||||
limit, then fails only at a real loading boundary or a small explicit buffer:
|
||||
32 KiB for the Codex chain, 20,000 characters for Hermes, and 250 effective
|
||||
Claude lines above the 200-line recommendation. The same envelope applies to the
|
||||
repo STS skill because `AGENTS.md` requires it on every task. A failure tells the
|
||||
repo Ponytail skill because `AGENTS.md` requires it for coding work. A failure tells the
|
||||
maintainer to use nested/path-scoped context, on-demand skills or linked docs,
|
||||
remove duplication and completed history, and delete stale conclusions; raising
|
||||
the ceiling is not the default remedy.
|
||||
|
||||
@@ -77,9 +77,10 @@ its `loopback-network` permission; without that permission the cross-origin
|
||||
request was blocked. The gate checks both expected outcomes.
|
||||
|
||||
A self-contained LAN room uses a localhost Host URL and a selected LAN IPv4
|
||||
Viewer URL backed by the same local TypeScript process. Its generated access
|
||||
password is persisted once and passed only to the Client's Host page in a
|
||||
consumed fragment. Explicit `--link` mode launches the packaged Cloudflare Quick
|
||||
Viewer URL backed by the same local TypeScript process. The Client persists an
|
||||
optional user-chosen access password and passes it only to its Host page in a
|
||||
consumed fragment; a blank value leaves the Local site open. Explicit `--link`
|
||||
mode launches the packaged Cloudflare Quick
|
||||
Tunnel against that same local HTTP server and supplies its random HTTPS origin
|
||||
to Node before startup. The public path therefore reuses the exact frontend,
|
||||
WebSocket signaling, RoomStore, and Viewer grant instead of adding a rendezvous
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# Browser NAT Traversal
|
||||
# Browser And Native NAT Traversal
|
||||
|
||||
Last reviewed: 2026-09-02
|
||||
|
||||
This document owns evidence for improving direct Browser ICE without adding a
|
||||
This document owns evidence for improving direct ICE without adding a
|
||||
new relay or a custom transport. The current product contract remains standard
|
||||
WebRTC ICE with STUN discovery and bounded SFU fallback.
|
||||
|
||||
@@ -56,21 +56,50 @@ supports an opt-in mechanism, not a default or a participant-wide classifier.
|
||||
|
||||
## Current Decision
|
||||
|
||||
The ordinary self-hosted endpoint remains the discovery service. NAT prediction
|
||||
is absent by default; a deployment may enable same-host STUN-only listeners on
|
||||
3479 and 3480. Enabled deployments expose a per-share Host switch that defaults
|
||||
on and remains locked while sharing.
|
||||
An ordinary STUN endpoint remains the discovery service. A Site may enable
|
||||
same-host STUN-only listeners on 3479 and 3480. Public-link Client mode supplies
|
||||
one ordinary public endpoint plus two public survey destinations; pure LAN mode
|
||||
supplies none. Available authorities expose the same per-share Host switch,
|
||||
which defaults on and remains locked while sharing.
|
||||
|
||||
The accepted adapter is deliberately connection-local and additive. It
|
||||
derives the two auxiliary ports from the existing STUN authority, waits for a
|
||||
The accepted adapter is deliberately connection-local and additive. It waits for a
|
||||
clear three-point arithmetic `srflx` shape in one ICE generation, and appends a
|
||||
small two-sided candidate window from the observed port-sequence endpoint.
|
||||
Ordinary candidates trickle immediately, so unavailable auxiliary listeners
|
||||
cannot hold back stock ICE. There is no NAT label, hard candidate skip,
|
||||
route-controller input, or SFU preference. The switch applies to Host, Viewer
|
||||
upstream, and Viewer relay P2P connections. Its exact scope is recorded in
|
||||
route-controller input, or SFU preference. Browser candidates are filtered by
|
||||
their reported STUN URL. Native Pion performs the same survey through its
|
||||
`UniversalUDPMux`, so every observation and subsequent media packet uses one
|
||||
socket. Only explicitly marked Native survey observations feed prediction;
|
||||
the independently mapped-port candidate does not. The switch applies to Host,
|
||||
Viewer upstream, and Viewer relay P2P connections. Its exact scope is recorded in
|
||||
[ADR-0009](../adr/0009-optional-nat-prediction.md).
|
||||
|
||||
### Native Shared-Socket Preflight
|
||||
|
||||
On 2026-09-05, the unmodified Pion srflx gatherer contacted three public STUN
|
||||
destinations from three different temporary local ports, despite the media
|
||||
engine's ordinary UDP mux. This invalidated the prior assumption that Native
|
||||
STUN and media already shared one socket. Pion ICE's `UniversalUDPMux` was then
|
||||
used as the media mux and direct STUN observation owner. Its emitted srflx
|
||||
candidate reported the exact Engine listener as its related port.
|
||||
|
||||
The same build carried a public-link session to an independent Linux Pion
|
||||
Viewer: 35 H.264 RTP packets arrived over a selected direct host-to-srflx pair,
|
||||
while the Quick Tunnel carried signaling only. The active TUN network yielded
|
||||
one distinct mapped endpoint across the public survey, so no port sequence or
|
||||
prediction was claimed. The result proves shared-socket discovery and transport,
|
||||
not public-survey availability or a predicted-path success rate.
|
||||
|
||||
Native Site and public-link shares also request one PCP, UPnP, or NAT-PMP
|
||||
mapping for that same socket. The returned port is advertised as a
|
||||
lower-priority candidate using a public address already observed by ordinary
|
||||
STUN. This is additive and bounded; a VPN, double NAT, or absent mapping service
|
||||
can make it unusable without delaying or replacing ordinary ICE.
|
||||
A local UDP-forwarding gate then withheld every ordinary Host candidate and
|
||||
connected Pion ICE/DTLS in 1.26 seconds through the advertised `mp1` endpoint.
|
||||
That proves the same-socket ICE mechanism, not rescue through a physical NAT.
|
||||
|
||||
### Independent Observation And Attribution Preflight
|
||||
|
||||
On 2026-09-01, Chrome 151 on the flagship Windows network gathered one `srflx`
|
||||
@@ -180,4 +209,5 @@ a new Peer route.
|
||||
- [libp2p Browser and DCUtR hole-punching boundary](https://github.com/libp2p/specs/blob/master/connections/hole-punching.md)
|
||||
- [coturn listener and auxiliary endpoint reference](https://github.com/coturn/coturn/blob/master/examples/etc/turnserver.conf)
|
||||
- [Cloudflare Realtime STUN service](https://developers.cloudflare.com/realtime/turn/)
|
||||
- [Pion Universal UDP mux](https://github.com/pion/ice/blob/main/udp_mux_universal.go)
|
||||
- [WebRTC selected candidate stats](https://www.w3.org/TR/webrtc-stats/#dom-rtcicecandidatestats-foundation)
|
||||
@@ -0,0 +1,120 @@
|
||||
# Native Client Lifecycle
|
||||
|
||||
- Reviewed: 2026-09-05
|
||||
- Scope: Windows capture idle semantics, native loopback input isolation, and
|
||||
Browser visibility of an unexpected Client disconnect.
|
||||
- Status: static-source, preview-queue and Client-crash checks pass; Windows 10
|
||||
monitor startup and game-specific window replacement remain unresolved.
|
||||
|
||||
## Current Evidence
|
||||
|
||||
An idle WGC source is not a capture-end signal. Source/target closure, control
|
||||
session loss and individual edge failures have separate owners. A Viewer may
|
||||
need a keyframe from the retained image while a source remains unchanged.
|
||||
|
||||
Source previews use the same serial control connection as share startup. A
|
||||
virtual-time reproduction with three three-second previews made the real
|
||||
`NativeClient.startShare()` time out at eight seconds, before its handler ran
|
||||
at nine seconds. The picker now sends one preview at a time, cancels unsent
|
||||
work when its source view is retired, and waits for the remaining preview before
|
||||
starting native media. Refresh and Client replacement fence old results.
|
||||
|
||||
## Scope Decisions
|
||||
|
||||
- Keep one strict loopback wire and one-share/one-session fence.
|
||||
- Recovery does not choose a different capture target, codec or media route.
|
||||
- Do not add a tunnel watchdog, local-process authentication scheme, custom
|
||||
quality score, or compatibility alias in this phase.
|
||||
|
||||
The loopback identifier boundary is now aligned at 8-256 bytes, and the link
|
||||
gate is included in TypeScript checking. Evidence wording distinguishes scripted
|
||||
package gates from physical-device gates. The possible cloudflared orphan and
|
||||
the intentionally per-user local trust boundary remain separate decisions; no
|
||||
runtime change for either is claimed here.
|
||||
|
||||
## Implemented Boundary
|
||||
|
||||
- Windows capture treats a quiet frame pool as idle. It retains one latest
|
||||
converted frame and emits it only when an explicit recovery keyframe request
|
||||
arrives; source-close and target-process signals remain the end conditions.
|
||||
- Native Pion candidate input is syntax-checked with the existing ICE parser,
|
||||
bounded before queuing, and discarded when malformed, stale, or over capacity.
|
||||
Repeated answers for an edge are idempotent. Profile updates remain fenced to
|
||||
that same share session in the current loopback v8 wire.
|
||||
- An unexpected Browser control-socket close now notifies the active Host owner;
|
||||
intentional user cleanup remains silent and uses the existing share fence.
|
||||
|
||||
## Physical Evidence
|
||||
|
||||
- On 2026-09-04, a blank Windows Notepad window was selected as a native source.
|
||||
The capture process stayed alive beyond seven seconds without a new source
|
||||
change, grew from 7,664 to 16,095 output bytes after a `K` request, and then
|
||||
exited cleanly. This validates the quiet-source and retained-frame path on the
|
||||
current Windows host; it does not claim a cross-version Windows matrix.
|
||||
- The same day, the local Native Host gate started a real Host/Viewer share,
|
||||
terminated the Client process, and observed the Host return to its start-share
|
||||
control within the bounded gate. Browser, capture, server, ports, and profile
|
||||
cleanup all passed.
|
||||
- On 2026-09-05, a Windows 11 build 26200 display-source check produced 91
|
||||
hardware-H.264 frames at the 720p30 profile and exited normally. A separate
|
||||
source-window check stayed alive through 7.02 seconds without frames while
|
||||
minimized, then produced 17 frames within 540 ms of restoration. All owned
|
||||
processes, ports and profiles were cleaned up. Neither check establishes the
|
||||
reported Windows 10 monitor-start failure or game HWND recreation as fixed.
|
||||
|
||||
## Open Capture Reports
|
||||
|
||||
The Windows 10 report concerns selecting one entire monitor; application/window
|
||||
capture, including fullscreen games, works. `CreateForMonitor` supports Windows
|
||||
10 1903 and later; the current single-HMONITOR call, identity checks and
|
||||
`CreateFreeThreaded` setup match the documented API. No missing general permission
|
||||
request has been established. Community reports of disabled capture services or
|
||||
invalid display contexts are leads, not a diagnosis of this user's machine.
|
||||
Projected WinRT exceptions now reach the existing capture-error boundary instead
|
||||
of escaping `std::exception`; that correction alone does not prove startup fixed.
|
||||
|
||||
The original converter always fitted the captured pixel aspect into the output:
|
||||
1280x960 into 1920x1080 yielded 1440x1080 content and 240-pixel side bars, even
|
||||
when Windows explicitly stretched that desktop source onto a 1920x1080 target.
|
||||
The Windows adapter now reads the active `QueryDisplayConfig` path and applies
|
||||
its target aspect only for explicit `STRETCHED`, a unique display path, exact
|
||||
source/frame dimensions, and display or physical full-client-area capture.
|
||||
Rotation changes the target aspect, not the orientation of already-oriented WGC
|
||||
pixels. Output profile dimensions and the one-encode media path are unchanged.
|
||||
|
||||
Display metadata refreshes on source geometry changes and existing periodic
|
||||
keyframes, without another timer or display-setting writes. Unknown/custom
|
||||
scaling, ambiguous cloned outputs and normal windows keep captured aspect.
|
||||
This does not prove fidelity to monitor-side scaling or vendor-private game
|
||||
panel-fit/MPO transforms: those may not appear in desktop display-path metadata.
|
||||
The CPU geometry check covers stretch, ordinary aspect, rotation and already-
|
||||
scaled input; the reported game's physical comparison remains acceptance work.
|
||||
|
||||
Windows explicitly permits a capture item to close when its application silently
|
||||
replaces the underlying window. Reattaching to another window is not equivalent
|
||||
to resuming the same selected target; this remains separate from the verified
|
||||
quiet/minimized-source behavior.
|
||||
|
||||
Primary references: [CreateForMonitor](https://learn.microsoft.com/en-us/windows/win32/api/windows.graphics.capture.interop/nf-windows-graphics-capture-interop-igraphicscaptureiteminterop-createformonitor),
|
||||
[CreateFreeThreaded](https://learn.microsoft.com/en-us/uwp/api/windows.graphics.capture.direct3d11captureframepool.createfreethreaded),
|
||||
[capture item closure](https://learn.microsoft.com/en-us/uwp/api/windows.graphics.capture.graphicscaptureitem.closed),
|
||||
and [WinRT exceptions](https://learn.microsoft.com/en-us/uwp/cpp-ref-for-winrt/error-handling/hresult-error).
|
||||
Aspect references: [active display paths](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-querydisplayconfig),
|
||||
[scaling modes](https://learn.microsoft.com/en-us/windows/win32/api/wingdi/ne-wingdi-displayconfig_scaling),
|
||||
[active signal size](https://learn.microsoft.com/en-us/windows/win32/api/wingdi/ns-wingdi-displayconfig_video_signal_info),
|
||||
and [independent panel fitting/MPO](https://learn.microsoft.com/en-us/windows/win32/direct3ddxgi/for-best-performance--use-dxgi-flip-model).
|
||||
|
||||
## Acceptance
|
||||
|
||||
- Static Windows source remains a live share for a bounded physical run, and a
|
||||
new Viewer can obtain a keyframe without a source change.
|
||||
- A malformed/stale ICE candidate or repeated per-edge answer does not close the
|
||||
control session or terminate sibling edges; a real edge lifecycle failure
|
||||
remains observable.
|
||||
- Killing the Client causes the active Host owner to leave the native share
|
||||
path promptly, while ordinary user-initiated cleanup does not recurse.
|
||||
- `npm run check` and the relevant Client checks pass; no credentials or raw
|
||||
media identifiers enter logs or documentation.
|
||||
|
||||
Cross-version capture and browser matrices remain separate acceptance work. They
|
||||
are not claimed by the current Windows gate.
|
||||
@@ -1,11 +1,10 @@
|
||||
# Native Client Media Evidence
|
||||
|
||||
- Reviewed: 2026-09-04
|
||||
- Scope: Windows capture, one shared H.264/Opus source, Pion transport, Browser
|
||||
- Reviewed: 2026-09-05
|
||||
- Scope: platform capture, one shared H.264/Opus source, Pion transport, Browser
|
||||
decode, and Browser-mediated SFU fallback
|
||||
- Status: native Host, cross-NAT video, Windows process/system audio, capture-failure
|
||||
restart, native P2P quality-evidence, and native-source SFU gates passed;
|
||||
other platform media remain outside the boundary
|
||||
- Status: Windows physical native Host and Viewer gates passed; macOS and Linux adapters
|
||||
compile and package but still require physical media gates
|
||||
|
||||
## Result
|
||||
|
||||
@@ -20,6 +19,15 @@ received 30 packets over a selected `srflx`-to-`srflx` pair. The validation
|
||||
session carried signaling through a temporary reverse SSH path only; media was
|
||||
negotiated directly by ICE.
|
||||
|
||||
On 2026-09-05, a separate Windows Local gate used a Browser Host with synthetic
|
||||
1280x720 motion and a Client-activated Viewer. The Viewer exclusively claimed
|
||||
the v8 loopback control session and real Chrome decoded 621 frames at 1280x720.
|
||||
This proves the Browser-to-Native receive/local-bridge path rather than a silent
|
||||
Browser fallback. A Pion integration gate separately forwards the same H.264
|
||||
and Opus payload from one inbound receiver source to a bounded downstream edge,
|
||||
while dropping upstream connection-local extensions so each outbound Pion
|
||||
interceptor writes its own negotiated TWCC header.
|
||||
|
||||
The one-link form carried the current signaling protocol through its temporary
|
||||
public WSS origin to an independent Linux Pion Viewer. Repeated runs delivered
|
||||
30+ packets over selected direct paths using a reflexive candidate; the Quick
|
||||
@@ -27,8 +35,9 @@ Tunnel carried no media.
|
||||
|
||||
An isolated LiveKit 1.13.6 gate carried the same native source through one
|
||||
reserved local Pion edge into the existing Browser `SfuPublisher`; an ordinary
|
||||
LiveKit Viewer then received 30 consecutive 1280x720 frames. The direct native
|
||||
P2P path, Browser, Client, LiveKit process, ports, and profiles all cleaned up.
|
||||
LiveKit Viewer received the default 1080p source, then 15 consecutive 854x480
|
||||
frames after one live Native/SFU profile change. The direct native P2P path,
|
||||
Browser, Client, LiveKit process, ports, and profiles all cleaned up.
|
||||
|
||||
The opt-in `gate:client-media` run proved, in order:
|
||||
|
||||
@@ -65,9 +74,24 @@ requires Go 1.25; Client CI uses Go 1.26.6.
|
||||
|
||||
## Current Boundary
|
||||
|
||||
The result does not yet prove live native quality-profile changes, macOS/Linux
|
||||
capture, or endurance. Native Host media is exposed only through the explicit
|
||||
Client-launched Host selection; these other capabilities remain unavailable there.
|
||||
Windows now supports hardware H264 and software VP8 behind the same capture,
|
||||
source and Pion transport boundary. libvpx 1.17.0 accepts the existing NV12 data
|
||||
with row pitch preserved, without another color-conversion library. In a local
|
||||
CPU-only 720p30 trial its mean encoding work was 6.112 ms per frame; this excludes
|
||||
GPU readback and is not game-quality evidence. Separate real Browser gates
|
||||
proved manual VP8 and Auto-selected H264, including live presets, pause/resume,
|
||||
source replacement and exact RTP codec preservation. Auto uses a bounded
|
||||
synthetic target-profile throughput check, not a score or a runtime codec switch.
|
||||
GPU-heavy selection and cross-device startup remain open acceptance work.
|
||||
|
||||
The Windows Browser gate now also keeps two native PeerConnections alive while
|
||||
the source changes from 720p30 to 1440p60, then changes to 480p15 while paused
|
||||
and resumes both Viewers. It proves the same Pion source survives two hardware
|
||||
capture/encoder generations; direct capture probes also produced every current
|
||||
resolution/FPS extreme, and the same route survives an explicit native source
|
||||
switch. The result does not yet prove macOS/Linux physical capture or endurance.
|
||||
Native Host media is exposed only through an explicit Client-launched Host
|
||||
selection; an ordinary Web Host retains Browser capture.
|
||||
|
||||
Native code does not publish directly to LiveKit. One local Pion edge gives the
|
||||
system Browser a remote `MediaStreamTrack`; WebRTC requires that remote track to
|
||||
@@ -77,6 +101,16 @@ The bridge adds one local decode for Host preview and a Browser encode only when
|
||||
SFU publication is active. Direct P2P children continue to reuse the one native
|
||||
H.264 encode. This preserves the accepted SFU behavior without another LiveKit
|
||||
SDK or media policy.
|
||||
|
||||
A shared encode cannot independently adapt one bitstream for unequal paths.
|
||||
Rather than lower every Native child, a quality operation originating from a
|
||||
persistently degraded Native sender edge may prepare one Browser WebRTC sender
|
||||
from the stable local bridge. The existing overlapping candidate comparison
|
||||
alone commits or rolls it back, and a later quality operation can return that
|
||||
edge to Native. Healthy Native edges keep sharing the hardware encode. This is
|
||||
implemented without a new route reason, timer, score, or representation ladder;
|
||||
controlled weak-path physical acceptance remains open.
|
||||
|
||||
Native P2P edges negotiate transport-wide feedback and use Pion's send-side GCC
|
||||
with its immediate no-op pacer. The pacer neither queues nor applies one edge's
|
||||
estimate to the shared encoder. Once real feedback and source frames exist, the
|
||||
@@ -103,8 +137,11 @@ added about 0.38 MiB to the stripped Windows Client. Mapping begins
|
||||
with the share, is awaited before the first PeerConnection, refreshes only when
|
||||
a later edge arrives after half the requested lease, and is removed with the
|
||||
engine. Failure is cached for that share and leaves ordinary ICE/STUN unchanged.
|
||||
This proves lifecycle and non-regression, not that a mapped candidate has yet
|
||||
rescued a pair that public STUN alone could not connect.
|
||||
The returned port is advertised once per observed public address as a
|
||||
lower-priority candidate. Only the three explicit same-socket survey candidates
|
||||
feed NAT prediction; the mapped candidate does not. This proves lifecycle and
|
||||
signaling, not that the mapped candidate has yet rescued a pair that public STUN
|
||||
alone could not connect.
|
||||
|
||||
The retained hardware fixture also changed one live NVIDIA MFT through
|
||||
`3 Mbps -> 1.5 Mbps -> 3 Mbps` without recreating it. Equal 120-frame phases
|
||||
@@ -120,34 +157,63 @@ loopback audio; a window uses WGC plus process-tree loopback when that actual
|
||||
capability probe succeeds. Preview failure is advisory and cannot tear down
|
||||
the control session.
|
||||
|
||||
A controlled 144 fps window on Windows build 26200 exposed two capture-cadence
|
||||
failures: WGC's zero `MinUpdateInterval` delivered 48 fps, and sampling from the
|
||||
last emitted timestamp reduced 48 fps input to 24 fps for a 30 fps profile.
|
||||
Using WGC's supported non-zero interval plus a phase-continuous output cadence
|
||||
delivered 29.99 fps and 60.00 fps from the same source. Older Windows versions
|
||||
without the optional session interface retain their existing WGC behavior.
|
||||
|
||||
A Browser-capture/Pion fanout gate exposed a separate RTP input defect:
|
||||
payload-empty padding was rejected and ended the video or audio reader. Chrome
|
||||
then fell to a 30 kbps video target even at zero media loss and about 1 ms RTT.
|
||||
Discarding padding kept the reader alive but introduced downstream sequence
|
||||
gaps, NACKs and reduced decoded cadence. Forwarding valid padding through Pion's
|
||||
existing `WriteRTP`, while excluding it from media-frame counts, restored two
|
||||
1080p Viewers to about 30 fps with zero packet loss. The same gate passed live
|
||||
quality, pause/resume, audio-presence source changes and Client-exit fallback.
|
||||
Native payload-capacity evidence does not require decoded dimensions; encoded
|
||||
receivers can supply real transport evidence without opening a decoder.
|
||||
|
||||
DevTools accepted weak-network emulation commands in this loopback run, but RTP
|
||||
traffic exceeded the requested limit without the requested loss. That run does
|
||||
not establish weak-network quality or automatic fallback performance.
|
||||
|
||||
Non-Windows capture keeps the existing process/frame boundary and replaces only
|
||||
the platform sidecar. The macOS candidate enumerates `SCShareableContent`, fences
|
||||
the platform sidecar. The macOS adapter enumerates `SCShareableContent`, fences
|
||||
the selected process/window generation, receives change-driven
|
||||
`CMSampleBuffer`s, and requires VideoToolbox constrained-baseline hardware H.264.
|
||||
It retains one latest pixel buffer so an existing PLI/FIR can encode a fresh IDR
|
||||
even while the screen is unchanged; capture timestamps drive the shared Pion RTP
|
||||
clock. A GitHub `macos-15` arm64 runner compiled the sidecar, created the same
|
||||
hardware-only encoder, and encoded an in-memory 420v frame into a validated
|
||||
`42c01f` SPS/PPS/IDR. ScreenCaptureKit permission, real capture, static-frame
|
||||
recovery, and endurance still require a physical Mac.
|
||||
`42c01f` SPS/PPS/IDR. The current adapter also emits ScreenCaptureKit application
|
||||
or display audio through the common PCM boundary. ScreenCaptureKit permissions,
|
||||
real video/audio capture, static-frame recovery, and endurance still require a
|
||||
physical Mac.
|
||||
|
||||
Linux native capture remains no-go for the current stage. The XDG ScreenCast
|
||||
Portal owns a user-selected session and restricted PipeWire file descriptor, not
|
||||
the Windows-style pre-enumerated window target or an encoded stream. A thin
|
||||
future Wayland-only gate may dynamically use the distribution's GStreamer for
|
||||
`pipewiresrc`, one bounded queue, one proved hardware H.264 element, `h264parse`,
|
||||
and `appsink`; it must not ship GStreamer, add `webrtcbin`, implement X11 capture,
|
||||
or pretend that the portal provides target-process audio. Without that system
|
||||
runtime, direct DMA-BUF import plus VAAPI/Vulkan encoding is not a small adapter.
|
||||
The Linux adapter lets the XDG ScreenCast Portal own source selection, consumes
|
||||
its restricted PipeWire stream through one bounded GStreamer pipeline, and
|
||||
requires an installed element classified as a hardware H.264 encoder. It emits
|
||||
the same Annex-B protocol and can use the PulseAudio-compatible default monitor
|
||||
for system audio; it does not claim per-process audio. GitHub Linux CI compiles,
|
||||
probes, packages, starts, and stops the candidate. A real Portal desktop,
|
||||
hardware encoder, audio source, Browser decode, and recovery still require a
|
||||
physical Linux gate. GStreamer stays a system dependency: bundling another RTC
|
||||
or an 80+ MB media runtime would defeat the thin-adapter boundary, while an
|
||||
unavailable dependency cleanly leaves Browser capture available.
|
||||
|
||||
## Implementation Boundary
|
||||
|
||||
- `nativecapture` owns the child process, source identity, and bounded frame protocol.
|
||||
- `mediaedge` owns the stable Pion API, one UDP mux, shared H.264/Opus sources,
|
||||
and independent PeerConnections.
|
||||
- `nativehost` composes one capture generation with its bounded edges.
|
||||
- `nativecontrol` maps only local source/share/edge commands and exact native
|
||||
sender quality windows to the loopback v5 wire.
|
||||
- `nativehost` owns the current capture generation and its bounded stable edges.
|
||||
- `nativeviewer` owns one native inbound H.264/Opus source and its encoded child
|
||||
edges; it does not own room or route state.
|
||||
- `nativecontrol` maps local source/share/receive/edge commands and exact native
|
||||
sender quality windows, live profile updates, and source replacement to the
|
||||
loopback v8 wire.
|
||||
|
||||
The deleted sender application, UI, room client, and old wire are not
|
||||
compatibility inputs. Historical measurements remain in the separately marked
|
||||
@@ -156,6 +222,8 @@ compatibility inputs. Historical measurements remain in the separately marked
|
||||
## Primary Sources
|
||||
|
||||
- [Windows Graphics Capture](https://learn.microsoft.com/en-us/windows/apps/develop/media-authoring-processing/screen-capture)
|
||||
- [WGC `MinUpdateInterval`](https://learn.microsoft.com/en-us/uwp/api/windows.graphics.capture.graphicscapturesession.minupdateinterval)
|
||||
- [Sunshine WGC update interval](https://github.com/LizardByte/Sunshine/blob/master/src/platform/windows/display_wgc.cpp)
|
||||
- [WGC `CreateForWindow`](https://learn.microsoft.com/en-us/windows/win32/api/windows.graphics.capture.interop/nf-windows-graphics-capture-interop-igraphicscaptureiteminterop-createforwindow)
|
||||
- [WASAPI process loopback](https://learn.microsoft.com/en-us/samples/microsoft/windows-classic-samples/applicationloopbackaudio-sample/)
|
||||
- [OBS application-audio capture guide](https://obsproject.com/kb/application-audio-capture-guide/)
|
||||
@@ -167,14 +235,17 @@ compatibility inputs. Historical measurements remain in the separately marked
|
||||
- [Pion no-op pacer RTX issue](https://github.com/pion/interceptor/issues/406)
|
||||
- [WebRTC remote-track constraints](https://www.w3.org/TR/webrtc/#mediastreamtrack-network-use)
|
||||
- [Pion single-port ICE](https://github.com/pion/webrtc/tree/master/examples/ice-single-port)
|
||||
- [Pion broadcast relay](https://github.com/pion/webrtc/tree/master/examples/broadcast)
|
||||
- [gopus pure-Go Opus codec](https://github.com/thesyncim/gopus)
|
||||
- [Tailscale port mapper](https://github.com/tailscale/tailscale/tree/main/net/portmapper)
|
||||
- [libp2p NAT port mapping](https://github.com/libp2p/go-libp2p/blob/master/options.go)
|
||||
- [NetBird standalone Go NAT](https://github.com/netbirdio/go-nat)
|
||||
- [WebRTC signaling and ICE](https://webrtc.org/getting-started/peer-connections)
|
||||
- [Apple ScreenCaptureKit](https://developer.apple.com/documentation/screencapturekit)
|
||||
- [ScreenCaptureKit audio output](https://developer.apple.com/documentation/screencapturekit/scstreamoutputtype/audio)
|
||||
- [ScreenCaptureKit idle frames](https://developer.apple.com/documentation/screencapturekit/scframestatus/idle)
|
||||
- [Apple VideoToolbox hardware encoder requirement](https://developer.apple.com/documentation/videotoolbox/kvtvideoencoderspecification_requirehardwareacceleratedvideoencoder)
|
||||
- [XDG ScreenCast Portal](https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.ScreenCast.html)
|
||||
- [libportal](https://libportal.org/libportal.html)
|
||||
- [PipeWire DMA-BUF contract](https://docs.pipewire.org/devel/page_dma_buf.html)
|
||||
- [GStreamer PipeWire source](https://gstreamer.freedesktop.org/documentation/pipewire/pipewiresrc.html)
|
||||
@@ -160,7 +160,8 @@ configuration checks and media evidence would still be required.
|
||||
This research uses W3C/IETF specifications, BSD-licensed libwebrtc source,
|
||||
public browser-compatibility data, and an official TeamSpeak technical
|
||||
statement. No GPL/AGPL implementation code was copied. GPL/AGPL projects remain
|
||||
study-only until the repository license and distribution model are decided.
|
||||
study-only unless a separate distribution decision accepts their obligations;
|
||||
the repository's own [MIT license](../../LICENSE) does not replace theirs.
|
||||
|
||||
## Primary Sources
|
||||
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
# Server Consolidation
|
||||
|
||||
- Reviewed: 2026-09-05
|
||||
- Status: design candidate for the phase after Browser/Client acceptance
|
||||
- Scope: one shared Go backend for Hosted and self-contained Client operation
|
||||
|
||||
## Current Ownership
|
||||
|
||||
Hosted [`index.ts`](../../src/server/index.ts) and Local
|
||||
[`local-index.ts`](../../src/server/local-index.ts) already compose the same
|
||||
[`createScreenerServer`](../../src/server/app.ts). The Go
|
||||
[`clientapp`](../../native/client/internal/clientapp/app.go) supervises that Node
|
||||
process and supplies native media; it does not duplicate room or route authority.
|
||||
Consolidation replaces this sole backend rather than merging two implementations.
|
||||
|
||||
[`RoomStore`](../../src/server/room-store.ts) owns room authority and participants;
|
||||
[`RoomDatabase`](../../src/server/room-database.ts) persists only access and lease
|
||||
material. [`RoomRouteController`](../../src/server/room-route-controller.ts)
|
||||
decides graph transitions without network I/O;
|
||||
[`HybridMediaRouter`](../../src/server/hybrid-media-router.ts) executes signaling
|
||||
and SFU effects. These are useful boundaries, not duplication to remove.
|
||||
|
||||
## Preparatory Simplification
|
||||
|
||||
1. Unify room termination cleanup. `SignalingServer.closeRoom` and `expireRooms`
|
||||
in [`signaling.ts`](../../src/server/signaling.ts) delete the same timers,
|
||||
connection evidence, share state and router resources before closing sockets.
|
||||
Use one cleanup owner with the existing termination reason as input.
|
||||
2. Remove the Hybrid connection-ID mirror. `connectionIdsByViewer` is populated
|
||||
through router callbacks, then `handleViewerQualityEvidence` checks that map
|
||||
and `resolveActiveViewerMediaEdge` again. Derive the Hybrid identity from the
|
||||
committed edge; retain the evidence gate's own last-observation identity.
|
||||
Cover reauthentication and in-place connection adoption before removing the
|
||||
callbacks. The ordinary routing mode remains a separate consumer until its
|
||||
product decision is made.
|
||||
|
||||
These are structural simplifications, not reproduced functional failures.
|
||||
|
||||
## Product Choice
|
||||
|
||||
`PEER_ASSISTED_MEDIA=false` remains a documented, default configuration in
|
||||
[`.env.example`](../../.env.example). It owns a separate Host-star path in
|
||||
`SignalingServer`, including child admission and SDP-answer readiness; Local and
|
||||
production use the graph controller. Decide whether this mode still belongs to
|
||||
the product before translating it. If automatic Peer topology is universal,
|
||||
remove the switch and ordinary path together; otherwise retain its explicit
|
||||
contract; this reachable mode is not dead code.
|
||||
|
||||
## Target Composition
|
||||
|
||||
```text
|
||||
one Go module
|
||||
cmd/screener-server -> shared server application
|
||||
cmd/screener-client -> shared server application + launcher + native media
|
||||
internal/server -> rooms, routing, HTTP/signaling, persistence, SFU adapters
|
||||
internal/client -> current native and platform boundaries
|
||||
one React/Vite static artifact
|
||||
```
|
||||
|
||||
Use packages for cohesive owners and files for local concerns. Do not introduce
|
||||
a framework layer per file, a second product API, or a multi-module build.
|
||||
Move the existing `native/client` Go module into the common scope during the
|
||||
migration. Hosted and Local construct the same validated application options;
|
||||
entry points own their defaults, reachability and lifecycle policy.
|
||||
|
||||
Each room keeps one serialized state owner. Go HTTP/WebSocket handlers must not
|
||||
mutate room, share and route maps independently. Run password derivation and
|
||||
LiveKit I/O outside that owner, then apply results only while the exact session,
|
||||
share and operation identities still match. Preserve the existing distinction
|
||||
between decisions and effects rather than replacing Node serialization with
|
||||
unrelated locks around each map.
|
||||
|
||||
## Reusable Components
|
||||
|
||||
- Use standard [`net/http`](https://pkg.go.dev/net/http) and
|
||||
[`embed`](https://pkg.go.dev/embed) for HTTP and built assets.
|
||||
- Reuse the Client's [`coder/websocket`](https://pkg.go.dev/github.com/coder/websocket)
|
||||
for signaling and the official [LiveKit Go SDK](https://github.com/livekit/server-sdk-go)
|
||||
for room control and tokens.
|
||||
- Keep the current SQLite schema and transactions. Compare a small
|
||||
[`modernc.org/sqlite`](https://pkg.go.dev/modernc.org/sqlite) build/runtime gate
|
||||
against package and locking needs before selecting a driver; no ORM is needed.
|
||||
- Follow [Go's shared `internal` and multiple `cmd` guidance](https://go.dev/doc/modules/layout).
|
||||
Retain current Pion, capture sidecars and control-tunnel ownership.
|
||||
|
||||
## Minimum Acceptance
|
||||
|
||||
Preserve exact Host/session replacement, grant revocation, atomic room replacement,
|
||||
SQLite restart continuity and Local shutdown semantics. Exercise one graph and
|
||||
one operation through Peer, SFU, same-edge regeneration, bounded overlap and
|
||||
rollback, including stale asynchronous completions. Preserve short-lived SFU
|
||||
authority and release resources only after physical drain confirmation.
|
||||
|
||||
Use the same Browser contract scenarios against Hosted and Local, plus one
|
||||
physical mixed Browser/Client media flow. Measure package size, startup and idle
|
||||
memory before and after. Keep one strict wire contract and one schema owner;
|
||||
avoid hand-maintained parallel TypeScript/Go validation rules. Retire the Node
|
||||
server, bundled runtime, supervisor and old packaging paths in the same accepted
|
||||
integration boundary, preserving [ADR-0005](../adr/0005-automatic-hybrid-media-routing.md)
|
||||
and [ADR-0010](../adr/0010-cross-platform-client-runtime.md).
|
||||
+70
-27
@@ -1,6 +1,6 @@
|
||||
# Current Status
|
||||
|
||||
Last updated: 2026-09-04
|
||||
Last updated: 2026-09-05
|
||||
|
||||
This is the current execution index. Git history owns completed timelines;
|
||||
[verification status](./verification-status.md) owns evidence boundaries.
|
||||
@@ -50,7 +50,7 @@ This is the current execution index. Git history owns completed timelines;
|
||||
|
||||
## Current Source
|
||||
|
||||
- Canonical source uses the strict `screener-v19` Browser/server contract.
|
||||
- Canonical source uses the strict `screener-v20` Browser/server contract.
|
||||
Optional SQLite stable authority, non-expiring local preferred code, atomic
|
||||
room replacement and the Host codec selector remain implemented; production
|
||||
selects stable storage. Graceful restart, crash, timeout and network loss use
|
||||
@@ -76,10 +76,12 @@ This is the current execution index. Git history owns completed timelines;
|
||||
sender windows retain the bounded Peer operation; failed exact candidates stay
|
||||
consumed until the sender or candidate opportunity changes. The pre-share gate
|
||||
defaults on and locks while sharing.
|
||||
- Source keeps NAT prediction deployment-gated and disabled by default. When a
|
||||
deployment enables it, Host Advanced settings shows a per-share switch that
|
||||
defaults on. Every Browser P2P role uses the same bounded, connection-local
|
||||
adapter with only the deployment's self-hosted 3478/3479/3480 STUN endpoints;
|
||||
- NAT prediction stays authority-gated and defaults on only when offered. Browser
|
||||
and Native P2P use one connection-local rule with Site or Public-Link survey
|
||||
endpoints; pure LAN, route scoring, and SFU preference remain unchanged.
|
||||
Enabled acquisition now shares a three-attempt budget in the route controller,
|
||||
across foreground P2P and background direct continuation. Viewer progress
|
||||
reflects actual candidate creation; quality trials retain their prior budget.
|
||||
candidate and selected-path logs retain anonymous
|
||||
`ordinary | predicted | unknown` provenance. Ordinary candidates and SFU
|
||||
fallback remain unchanged.
|
||||
@@ -94,7 +96,10 @@ This is the current execution index. Git history owns completed timelines;
|
||||
visible/resume/pageshow rebaseline and rearm current-frame proof. Host and
|
||||
Viewer SFU clients disable LiveKit page-leave auto-disconnect, so SFU recovery
|
||||
cannot stop Host-owned capture. Native Viewer controls still own playback,
|
||||
and manual reconnect stays on the current route.
|
||||
and manual reconnect stays on the current route. A Client-launched Viewer can
|
||||
receive H.264/VP8 and Opus through a native encoded source, bridge one local Browser
|
||||
preview, and reuse that source for compatible P2P children; unsupported codec
|
||||
or failed bridge returns to the Browser peer.
|
||||
- Viewer presentation derives access, Host, route, playback, and runtime state
|
||||
from separate owners. An exact media generation plus monotonic proof epoch
|
||||
owns visible-frame truth; every invalidation rearms proof, while stale epochs
|
||||
@@ -121,35 +126,73 @@ This is the current execution index. Git history owns completed timelines;
|
||||
including bounded same-edge connection regeneration. Both default it on with a
|
||||
pre-share Host opt-out. Weighted/global optimization remains parked.
|
||||
- Source contains the cross-platform Client outside the current Web release.
|
||||
One Go entry opens a lightweight system-Browser launcher for Local, temporary
|
||||
public-link, or Site operation, supervises the same Node application when
|
||||
needed, and owns loopback native media. The same Host UI explicitly selects
|
||||
Browser capture or an exact native screen/window. Windows gates prove
|
||||
One Go entry starts a process-level loopback capability service, then opens the
|
||||
current system-Browser launcher for Local, temporary public-link, or saved-Site
|
||||
operation. The saved Site remains allowed while another room source runs; the
|
||||
same Host UI explicitly selects
|
||||
Browser capture or an exact native screen/window. Windows Native now shares
|
||||
the VP8/Auto/H264 selector, with libvpx software VP8 and hardware H264; Auto
|
||||
uses a bounded target-profile throughput check. Both VP8 and Auto-selected
|
||||
H264 passed real Browser delivery and codec-stable live/source changes.
|
||||
Windows gates prove
|
||||
packaged Local/Site operation, WGC hardware-H.264 plus process/system audio, bounded
|
||||
shared-encode P2P, and cleanup. The window arm also proves source end and
|
||||
same-room reselection; display-source lifecycle remains a separate physical
|
||||
gate. An explicit target assembler also produces a
|
||||
Linux amd64 package with proved Local/one-link runtime and a structurally
|
||||
verified but unexecuted macOS arm64 package; its capture sidecar now passes an
|
||||
arm64 compile plus hardware-H.264 IDR self-test. Client-scoped pull requests
|
||||
build and Local-smoke all three candidates before merge; a validated `main`
|
||||
push emits them as short-lived Actions artifacts. One-link mode preserves the Host Local
|
||||
authority while exposing its ordinary invitation and HTTP/WebSocket control
|
||||
path through a session-scoped Quick Tunnel. An independent Linux Pion Viewer
|
||||
verified macOS arm64 package. Its capture sidecar passes an arm64 compile,
|
||||
hardware-H.264 IDR self-test, and audio-adapter build, but the package and
|
||||
capture path remain unrun on a physical Mac. The Linux package now includes a
|
||||
thin Portal/PipeWire/GStreamer hardware-H.264 capture adapter and passes its
|
||||
compile, probe, package, startup, and shutdown checks; real desktop capture,
|
||||
system audio, and recovery remain unproved. Client-scoped pull requests build
|
||||
and run the scripted Local smoke on all three candidates before merge; a
|
||||
validated `main` push emits them as short-lived Actions artifacts. One-link
|
||||
mode preserves the Host Local authority while exposing its ordinary invitation
|
||||
and HTTP/WebSocket control path through a session-scoped Quick Tunnel. An
|
||||
independent Linux Pion Viewer
|
||||
has received that native media over a direct ICE pair; remote Browser media
|
||||
remains unproved. Site and one-link native shares also make one bounded
|
||||
best-effort PCP/UPnP/NAT-PMP mapping for their sole UDP socket without
|
||||
changing ICE when no gateway accepts it; pure LAN Local mode does not.
|
||||
best-effort PCP/UPnP/NAT-PMP mapping and Pion Universal-UDP-mux STUN discovery
|
||||
on their media socket. A public-link run delivered 35 H.264 packets to an
|
||||
independent Linux Viewer; no predicted-path win is yet claimed.
|
||||
Native P2P edges now negotiate TWCC and send Pion GCC's target-versus-source
|
||||
payload category through the existing sender-quality evidence path without
|
||||
pacing or changing the shared encode. A local bridge now supplies native Host
|
||||
media to the existing Browser LiveKit publisher; an isolated LiveKit gate
|
||||
proves 1280x720 delivery and complete cleanup without a second SFU client.
|
||||
Non-Windows capture remains outside the release. The default packaged Client
|
||||
launcher performs a non-blocking official GitHub Release check and the
|
||||
deployment tree provides a read-only operator check against the same full-SHA
|
||||
release identity; neither installs or interrupts a running share.
|
||||
pacing or changing the shared encode. The existing quality operation may test
|
||||
a Browser sender for one persistently degraded Native edge without lowering
|
||||
healthy shared edges. A local bridge supplies native Host media to the existing
|
||||
Browser LiveKit publisher; an isolated LiveKit gate
|
||||
proves default 1080p delivery, a live change to 480p, and complete cleanup
|
||||
without a second SFU client.
|
||||
A Windows gate proves Browser Host to Client-native Viewer delivery and Chrome
|
||||
decode; an RTP gate covers bounded H.264/Opus child relay and cleanup.
|
||||
Client-assisted Browser H.264 capture now reuses one local sender for Native
|
||||
fanout while quality convergence is enabled. Its two-Viewer gate covers
|
||||
cadence, live controls and Client-exit recovery; weak-network acceptance
|
||||
remains open. Native RTP forwarding preserves padding and its sequence
|
||||
continuity without requiring decoded dimensions for capacity evidence.
|
||||
Native control treats malformed/stale/repeated per-edge signaling as disposable;
|
||||
unexpected Client close reaches the current Browser participant through the
|
||||
existing fence. Windows
|
||||
retains one converted frame for quiet-source keyframe recovery. Its live
|
||||
profile gate keeps two PeerConnections while moving the hardware source from
|
||||
720p30 to 1440p60, then to 480p15 while paused, and resumes both Viewers;
|
||||
current source-switch, static, and crash gates pass, while cross-version
|
||||
capture remains open.
|
||||
Local Client access is open by default and can be protected with a user-chosen
|
||||
password from the Browser launcher; Hosted Site access is unchanged.
|
||||
Non-Windows native media remains outside physical acceptance. The default
|
||||
packaged Client launcher performs a non-blocking official GitHub Release check;
|
||||
the deployment tree provides a read-only operator check against the same full-
|
||||
SHA release identity. Neither installs or interrupts a running share.
|
||||
|
||||
## Current Hold
|
||||
|
||||
No source or deployment P0/P1 is open; sanitized route logging remains enabled.
|
||||
Client acceptance remains open for the reported Windows 10 display startup and
|
||||
game-specific source behavior. Windows 11 display, minimized-source recovery,
|
||||
live presets and source-switch checks pass, but do not establish those reports
|
||||
as resolved. NAT acquisition has controller and signaling coverage, not a new
|
||||
public-network success-rate claim. Its added prepare progress field requires a
|
||||
coordinated private-wire release: old strict pages cannot consume it safely.
|
||||
Native codec and capability fields likewise require the matching bundled UI.
|
||||
Production is unchanged; [TODO](./todo.md) owns acceptance and release work.
|
||||
+45
-60
@@ -1,44 +1,39 @@
|
||||
# Current TODO Ledger
|
||||
|
||||
Last reviewed: 2026-09-04
|
||||
Last reviewed: 2026-09-05
|
||||
|
||||
Only **Now** is executable. Observations, old branches, experiments, and parked
|
||||
topics are not implementation authority.
|
||||
|
||||
## Now
|
||||
|
||||
1. **Native Client media boundary.** Windows native Host video now runs through
|
||||
the current Browser-owned room and route contract. Local Client startup,
|
||||
hardware-H.264 capture, bounded Pion fanout, and a remote `srflx`-to-`srflx`
|
||||
video gate now pass. Process-loopback audio passes the Windows Browser gate
|
||||
on the same PeerConnection with a video-only fallback. Pion TWCC/GCC now
|
||||
supplies exact native P2P sender-quality evidence through the existing route
|
||||
windows without a custom score. A reserved local bridge also passes the
|
||||
native-source SFU gate through the existing Browser LiveKit publisher and
|
||||
its existing representation policy.
|
||||
Capture-source failure now has a physical end/restart gate through the same
|
||||
room and Viewer. Clean-revision Windows and Linux packages pass their Local
|
||||
runtime gates; Linux also passes one-link startup and shutdown. The macOS
|
||||
arm64 sidecar passes hosted compilation and a synthetic VideoToolbox hardware-
|
||||
H.264 IDR gate; physically run ScreenCaptureKit capture and recovery on macOS.
|
||||
Linux retains Browser capture. Reopen a
|
||||
Wayland-only Portal/PipeWire sidecar only with a real desktop/GPU gate and a
|
||||
decision to reuse system GStreamer without bundling it; do not add a second
|
||||
RTC, X11 capture stack, or hand-built DMA-BUF/encoder matrix.
|
||||
The system-Browser launcher now owns Local, temporary public-link, and Site
|
||||
selection, and the Host explicitly chooses Browser capture or an exact
|
||||
Client-owned screen/window without command-line target input. Its Windows
|
||||
physical gates pass window and display capture, bounded source previews,
|
||||
process/system audio, and restored Viewer delivery. The window arm also
|
||||
proves source end and same-room reselection; display-source lifecycle remains
|
||||
a separate physical gate.
|
||||
No-Site Internet mode now has one ordinary public invitation link that retains
|
||||
the Host's Local authority and passes remote control-path and independent
|
||||
Linux Pion media gates. Prove decoded one-link Browser media on a physical
|
||||
second device and a selected mapped path on a pair that public STUN alone
|
||||
cannot connect. Port mapping is implemented and physically creates/releases
|
||||
a router mapping, but it cannot replace an SFU/TURN fallback for a restricted
|
||||
pair.
|
||||
1. **Client physical acceptance.** Physically validate macOS and Linux
|
||||
capture/audio/recovery, a decoded public-link Browser Viewer, and a NAT case
|
||||
rescued beyond ordinary STUN, plus one mixed Browser/Native Viewer relay.
|
||||
2. **Client owner acceptance.** Review the localized terminal, Browser capture
|
||||
through Native fanout, live controls and Client-exit recovery before merging.
|
||||
Confirm weak-path convergence with effective media throttling; a DevTools
|
||||
command succeeding without limiting RTP is not a weak-path pass.
|
||||
3. **Capture startup and controls.** Diagnose Windows 10 selecting an entire
|
||||
display failing to start while window capture succeeds; do not conflate it
|
||||
with exclusive-fullscreen game capture. Current Windows 11 checks cover
|
||||
display startup, minimized-source recovery and live presets. Verify remaining
|
||||
game HWND/device-loss behavior and physically compare the implemented Windows
|
||||
display-aspect correction with the reported stretched game, including GPU
|
||||
scaling not exposed by the active display path. Compare bitrate changes against
|
||||
same-content delivery before changing encoder settings.
|
||||
4. **NAT acquisition and release acceptance.** Validate the implemented bounded
|
||||
three-attempt owner on restricted Browser/Native pairs; a new connection does
|
||||
not guarantee a different NAT mapping. The added prepare progress field is
|
||||
incompatible with old strict pages. Obtain owner approval for the coordinated
|
||||
Web/Server/Client update and its active-share impact before merging/deploying;
|
||||
do not mix the new field into an ongoing old-version share.
|
||||
5. **Native codec acceptance.** Windows Native VP8 and Auto-selected H264 now
|
||||
pass real Browser playback, live presets, paused changes and source-switch
|
||||
checks while preserving the selected codec. Validate GPU-heavy Auto selection
|
||||
and the tight four-second probe/five-second startup budget on other hardware;
|
||||
do not claim throughput screening guarantees perceptual quality or later
|
||||
load. Other native platform capture encoders remain H264.
|
||||
|
||||
## Parked Product Work
|
||||
|
||||
@@ -58,22 +53,10 @@ topics are not implementation authority.
|
||||
Auto or move an unproved result into the availability path. Validate whether
|
||||
H.264 qualification can choose a slower VP8 fallback under mixed GPU/CPU load
|
||||
before adding comparative or dual-codec probing.
|
||||
3. **Browser encoded-frame reuse.** After the current Browser experience release,
|
||||
gate one narrow video-edge adapter that preserves signaling, routing,
|
||||
ICE/TURN, DTLS, DataChannel congestion control, upstream WebRTC playback and
|
||||
SFU fallback while reusing one encoded representation across every compatible
|
||||
P2P descendant. First determine whether current Browser APIs or a mature
|
||||
module can preserve per-edge adaptation without re-encoding; do not accept a
|
||||
standard-WebRTC weak-edge fallback as the default merely because it is easy.
|
||||
Compare one layered SVC encode with a small room-wide simulcast set before any
|
||||
per-edge encoder; select layers from current edge capacity rather than
|
||||
classifying Viewers into permanent strong/weak groups.
|
||||
A strong/weak two-child gate must keep the strong path at full quality,
|
||||
recover the weak path without globally lowering the shared representation,
|
||||
bound queues and temporary copies, and measure CPU, bytes, latency, keyframe
|
||||
recovery, A/V timing and supported mobile Browsers. Reuse existing quality
|
||||
evidence and route operations; add no custom score, global worst-viewer rule,
|
||||
periodic ladder or unproved codec representation.
|
||||
3. **Browser encoded reuse.** Pure Browser operation keeps normal WebRTC
|
||||
senders; users can use Client to reduce repeated encoding. Revisit Browser
|
||||
reuse only for a small mature adapter that preserves adaptation and media
|
||||
quality without an application-owned transport or layer controller.
|
||||
4. **Control-plane resource fairness.** Calibrate one deployment-wide LiveKit
|
||||
control queue and simple HTTP/WebSocket ingress budgets before implementation;
|
||||
prove a self-hosted stale-token revocation boundary before independently
|
||||
@@ -116,10 +99,10 @@ topics are not implementation authority.
|
||||
Keep injected candidates additive and diagnostic; do not make them a
|
||||
participant-wide NAT label, hard candidate skip, route score, or SFU
|
||||
preference. Validate selected predicted paths and bounded resource impact
|
||||
before recommending it as a default for other deployments. Direct
|
||||
convergence currently consumes each deferred Peer parent once; add no
|
||||
cross-generation retry budget until field evidence proves that fresh ICE
|
||||
generations expose a repeatable prediction opportunity.
|
||||
plus one selected Native mapped-port path that fails with STUN alone before
|
||||
recommending either as a default for other deployments. Measure benefit of
|
||||
the implemented three-attempt budget before increasing it or claiming that
|
||||
independent connection attempts are independent NAT mappings.
|
||||
14. **Unresolved route-state ownership claims.** Reopen lower-revision
|
||||
reauthentication, active SFU failure during an unrelated prepare, and
|
||||
multi-child relay-evidence ownership only from an exact current-wire
|
||||
@@ -141,10 +124,12 @@ topics are not implementation authority.
|
||||
boundaries are complete. First measure package, startup, and maintenance
|
||||
gains. If accepted, migrate Hosted and Local together and delete the Node
|
||||
server in the same boundary; do not create or retain two room, signaling,
|
||||
persistence, or route-controller implementations.
|
||||
|
||||
## Decision Needed
|
||||
|
||||
- Choose the repository and distribution license before public release or
|
||||
package distribution. GPL/AGPL implementations remain research-only until
|
||||
then.
|
||||
persistence, or route-controller implementations. The
|
||||
[consolidation draft](./research/server-consolidation.md) identifies the two
|
||||
preparatory simplifications, the routing-mode product choice, and the shared
|
||||
module and concurrency model; implementation follows Client acceptance.
|
||||
17. **Opt-in diagnostics.** Replace ad hoc console logging with an explicit
|
||||
Client/Server debug option and bounded diagnostic export. Include revision,
|
||||
runtime state and sanitized capture/connection events; exclude credentials,
|
||||
media and raw process memory by default. A compressed feedback bundle needs
|
||||
deliberate collection and retention, not automatic uploads.
|
||||
+57
-13
@@ -1,6 +1,6 @@
|
||||
# Verification Status
|
||||
|
||||
Last updated: 2026-09-04
|
||||
Last updated: 2026-09-05
|
||||
|
||||
This file owns only cross-module physical evidence that still changes how the
|
||||
current product may be described. [Status](./status.md) owns exact source and
|
||||
@@ -77,21 +77,26 @@ address selection, Browser launch commands, bounded Node supervision, its
|
||||
loopback port/Origin/Host/session contract, and exact package revision. The same
|
||||
source cross-builds for Windows amd64, macOS arm64, and Linux amd64.
|
||||
|
||||
On Windows, Chrome for Testing 151.0.7922.138 completed the built Local page's
|
||||
On Windows, Chrome 151.0.7922.175 completed the built Local page's
|
||||
fragment access, Host surface, LAN invitation, and clean Client/Node/Browser
|
||||
shutdown. A separate synthetic run formed a P2P-only Host-plus-three-Viewer tree
|
||||
with one Browser relay and advancing frames at every Viewer. The Hosted Site
|
||||
completed loopback `hello`/`ping` after CDP granted `loopback-network`; without
|
||||
completed v8 loopback `hello`/`ping` after CDP granted `loopback-network`; without
|
||||
that permission Chrome blocked it as expected.
|
||||
The exact clean-revision Windows package assembled with Node 24.19.0 passed both
|
||||
Client gates; changing `app/REVISION` made it fail before opening a listener.
|
||||
The current clean-revision Windows package assembled with Node 24.19.0 passed the
|
||||
package checks recorded for that revision; changing `app/REVISION` made it fail
|
||||
before opening a listener. Its fresh Site loopback gate closed the Client,
|
||||
Browser, port, and temporary profile.
|
||||
The same explicit-target assembly produced a Linux amd64 package that ran its
|
||||
bundled Node application and loopback runtime on an independent Ubuntu host.
|
||||
Its full package also created a public link reachable from another network and
|
||||
closed that link and all local ports on exit. The macOS arm64 output contains
|
||||
matching Mach-O arm64 Client and Node binaries; its capture sidecar compiles on
|
||||
the macOS runner and passes the synthetic hardware-H.264 IDR self-test, but the
|
||||
full package and ScreenCaptureKit path have not run on a physical Mac.
|
||||
matching Mach-O arm64 Client and Node binaries. Its prior fixed-profile capture
|
||||
baseline and current variable-profile/audio sidecar compile on the macOS runner;
|
||||
the latter passes the synthetic hardware-H.264 IDR self-test. The full package
|
||||
and ScreenCaptureKit video/audio path have not run on a physical Mac. Linux CI
|
||||
also compiles, probes, packages, starts, and stops its Portal/PipeWire/system-
|
||||
GStreamer hardware-H.264 adapter; it has not captured on a physical desktop.
|
||||
|
||||
Still required are physical second-device LAN playback, first-run LNA prompt,
|
||||
no-STUN mDNS behavior, macOS package execution, and equivalent non-Windows
|
||||
@@ -105,16 +110,24 @@ verifies decoded audio energy. The native Host gate passes display and window
|
||||
source selection and bounded previews; its window arm closes the captured
|
||||
source, observes the current share end, restarts capture in the same room, and
|
||||
requires the existing Viewer to receive a different media object plus 30 new
|
||||
frames. Display-source lifecycle is not claimed by that arm.
|
||||
frames. The same product gate changes a live Native Host from default 1080p30
|
||||
to 1440p30, then changes resolution while paused and resumes the same Viewer
|
||||
media object through the Host UI. A separate two-Viewer media gate changes
|
||||
720p30 to 1440p60, then changes to 480p15 while paused and resumes both
|
||||
connections. Display-source lifecycle is not claimed by that arm.
|
||||
Native P2P quality evidence and the Browser-mediated native-source SFU happy
|
||||
path now have bounded gates. Production package integration, SFU recovery and
|
||||
endurance, and physical non-Windows capture remain unproved.
|
||||
path now have bounded gates. A focused unit gate proves that a quality candidate
|
||||
from a current Native sender edge selects the stock Browser sender, while ordinary
|
||||
candidates remain Native. Controlled weak-path commit/rollback, production
|
||||
package integration, SFU recovery and endurance, and physical non-Windows
|
||||
capture remain unproved.
|
||||
|
||||
A five-second headless topology run with 20 Viewers passed the existing route
|
||||
capacity, no-orphan-publication, quality-propagation, and decoded-frame checks;
|
||||
it is topology evidence, not a long-running or public-network quality claim.
|
||||
An isolated LiveKit Server 1.13.6 run also delivered 30 decoded 1280x720 native
|
||||
frames through the existing Browser SFU publisher and completed cleanup.
|
||||
An isolated LiveKit Server 1.13.6 run also delivered default 1080p native media,
|
||||
then 15 decoded 854x480 frames after a live source/publisher profile change,
|
||||
through the existing Browser SFU publisher and completed cleanup.
|
||||
|
||||
The one-link gate started a session-scoped public origin from a Windows Client,
|
||||
confirmed that its ordinary invitation used that origin, and served the Viewer
|
||||
@@ -124,6 +137,27 @@ Pion Viewer; repeated runs delivered 30+ H.264 RTP packets over selected direct
|
||||
paths using a reflexive candidate. Client exit stopped Node and the public link.
|
||||
This still does not prove decoded Browser media on a physical second device.
|
||||
|
||||
On 2026-09-05, a local Windows gate proved a Browser Host feeding a Client-
|
||||
activated Native Viewer: the Viewer held the v8 control session while Chrome
|
||||
decoded 621 frames at 1280x720. A separate Pion integration gate proves that the
|
||||
same inbound source forwards H.264 and Opus RTP to one downstream edge without
|
||||
re-encoding. A physical mixed-device relay and public-link Browser Viewer remain
|
||||
open.
|
||||
|
||||
On 2026-09-05, a Native preflight showed that Pion's ordinary srflx gatherer
|
||||
used three temporary local ports for three public STUN destinations. The current
|
||||
Universal UDP mux adapter instead emitted an srflx observation related to the
|
||||
Engine's sole media port. A public-link run then delivered 35 H.264 RTP packets
|
||||
from that Windows Native Host to the independent Linux Pion Viewer over a
|
||||
selected direct host-to-srflx pair. The active TUN path exposed only one distinct
|
||||
mapping, so this proves shared-socket discovery and transport, not prediction.
|
||||
|
||||
On 2026-09-04, a bounded Windows physical check held a static Notepad source
|
||||
open beyond seven seconds, produced a requested recovery keyframe from its
|
||||
retained image, and exited cleanly. The Native Host crash gate also terminated the
|
||||
Client process and observed the Host return to its start-share control within
|
||||
the bounded check. These results cover the current Windows build only.
|
||||
|
||||
The same clean revision assembled into a self-contained Windows package with a
|
||||
matching Node runtime, application tree, and native capture process. Chrome 151
|
||||
passed the package's Local startup gate and both Site loopback-permission arms;
|
||||
@@ -131,8 +165,18 @@ revision mismatch remains fail-closed before a listener starts.
|
||||
|
||||
## Interpretation Rules
|
||||
|
||||
The current Windows Native candidate additionally passed manual VP8 and Auto-
|
||||
selected H264 playback, with actual Host/Viewer RTP codec checks across live
|
||||
presets, paused changes and source replacement. A three-attempt NAT acquisition
|
||||
has deterministic controller/signaling coverage, not a field success-rate
|
||||
claim. The completed checks do not resolve the reported Windows 10 monitor
|
||||
startup or game-specific aspect/window-replacement issues. The matching bundle
|
||||
must be accepted before any coordinated private-wire deployment.
|
||||
|
||||
- Configuration, unit tests, loopback, and synthetic signaling prove invariants,
|
||||
not target-network or device behavior.
|
||||
- An automated/package gate is evidence from its scripted runner; a physical
|
||||
gate requires the named target device, network, or desktop environment.
|
||||
- Missing RTCStats fields are unknown, not zero.
|
||||
- Requested quality values are ceilings; actual sender/receiver stats are truth.
|
||||
- A page-lifecycle correction is recovery logic, not a keepalive guarantee.
|
||||
|
||||
Reference in new issue
Block a user