From 69f20e86e787c235cd6272dfa1b889ee39759932 Mon Sep 17 00:00:00 2001 From: AAA <35992542+TNTcraftHIM@users.noreply.github.com> Date: Mon, 31 Aug 2026 03:27:55 +0800 Subject: [PATCH] docs: document browser platform output boundaries (#353) --- docs/README.md | 2 + docs/research/platform-output.md | 92 ++++++++++++++++++++++++++++++++ docs/todo.md | 10 ++-- 3 files changed, 101 insertions(+), 3 deletions(-) create mode 100644 docs/research/platform-output.md diff --git a/docs/README.md b/docs/README.md index a69c231..13b1a3a 100644 --- a/docs/README.md +++ b/docs/README.md @@ -60,6 +60,8 @@ Media and platform: - [Realtime quality and codecs](./research/realtime-quality-adaptation.md) - [Browser background capture](./research/browser-background-capture.md) - [Browser screen audio](./research/browser-screen-audio-quality.md) +- [Browser NAT traversal](./research/nat-traversal.md) +- [Browser platform output](./research/platform-output.md) - [Native sender and shared encode](./research/native-sender.md) Rooms and repository practice: diff --git a/docs/research/platform-output.md b/docs/research/platform-output.md new file mode 100644 index 0000000..c905304 --- /dev/null +++ b/docs/research/platform-output.md @@ -0,0 +1,92 @@ +# Browser Platform Output + +Last reviewed: 2026-08-31 + +This document owns evidence for showing a live Screener session on an AirPlay, +Cast, or Presentation API display. It does not add a product output surface. + +## Current Decision + +Do not add an application Cast or AirPlay button yet. Browser or operating-system +tab/screen mirroring is already usable without Screener integration. The Web +platform does not give the current `video.srcObject = MediaStream` a portable +remote-playback contract. + +The only application design worth a physical prototype is a custom Cast +receiver acting as another authenticated Screener Viewer. It would receive its +own WebRTC route and count toward room and endpoint capacity; it would not reuse +or fling the controlling Browser's decoded video element. + +## Three Different Capabilities + +### System Or Tab Mirroring + +Browser and operating-system UI can mirror a tab, window, screen, or media +element. The W3C Remote Playback model explicitly permits this implementation +shape but does not standardize its transport, latency, quality, lifecycle, or +device coverage. Screener needs no adapter for it and cannot claim its result. + +### Media Remoting Or Flinging + +Remote Playback selects a compatible source from an HTML media element's +availability source set. The specification never defines `MediaStream` or +`srcObject` as a remote playback source. Local Media Capture support for +rendering a `MediaStream` in an HTML media element does not create a cross-device +source. + +The product implementations have the same boundary: + +- WebKit documents that AirPlay needs a unique playback URL; even local MSE/MMS + data needs a separate HTTP/HLS alternative. +- Google's default and styled Web Receivers load a media URL and document + containers plus progressive HTTP, HLS, DASH, and SmoothStreaming. Their + supported-media contract does not include a WebRTC `MediaStream`. + +Creating an HLS, DASH, MSE, MediaRecorder, or WebCodecs bridge would introduce a +second media plane, buffering, packaging, server delivery, codec policy, and +cleanup. It is not a small output adapter for the current low-latency route. + +### Receiver As A Viewer + +The Presentation API can ask a receiving user agent to load a presentation URL +and provides two-way messaging between the controlling and receiving browsing +contexts. A Google Custom Web Receiver is likewise a registered, hosted HTML5 +application with custom messaging. + +That makes a narrow architecture possible: launch a receiver page, deliver a +short-lived room-scoped Viewer grant over the established control channel, and +let that page join through the normal Screener Viewer and routing contracts. The +grant must not appear in the presentation URL, logs, storage, or device metadata. + +This is not accepted product behavior. Google's documented media matrix does +not promise WebRTC in a Custom Web Receiver, the Presentation API is not a +cross-Browser surface, and no target receiver has proved Screener's codec, +autoplay, audio, network, or lifecycle requirements. + +## Physical Acceptance Gate + +A prototype needs a named sender Browser and physical receiver. It must prove: + +- receiver-page HTTPS/WSS, `RTCPeerConnection`, H.264/VP8 negotiation, decoded + progress, screen audio, autoplay, and user-visible failure; +- end-to-end latency, cadence, resolution, recovery, pause/resume, and receiver + teardown on Cast session end; +- exact room admission, one Viewer identity, one route, endpoint-cap accounting, + and no credential in URLs or retained receiver state; and +- comparison with built-in tab/screen mirroring on the same sender and display. + +Until that gate exists, feature detection alone cannot justify a control. A +native helper, custom encoded transport, or server restream is not authorized by +this research. + +## Primary Sources + +- [Remote Playback API](https://www.w3.org/TR/remote-playback/) +- [Media Capture and Streams](https://www.w3.org/TR/mediacapture-streams/) +- [Presentation API](https://www.w3.org/TR/presentation-api/) +- [Google Web Receiver overview](https://developers.google.com/cast/docs/web_receiver) +- [Google Custom Web Receiver](https://developers.google.com/cast/docs/web_receiver/basic) +- [Google Cast supported media](https://developers.google.com/cast/docs/media) +- [Google Cast registration](https://developers.google.com/cast/docs/registration) +- [WebKit MSE/MMS and AirPlay](https://webkit.org/blog/15036/how-to-use-media-source-extensions-with-airplay/) +- [Apple AirPlay overview](https://developer.apple.com/library/archive/documentation/AudioVideo/Conceptual/AirPlayGuide/Introduction/Introduction.html) diff --git a/docs/todo.md b/docs/todo.md index df57cd6..69ebefe 100644 --- a/docs/todo.md +++ b/docs/todo.md @@ -52,13 +52,17 @@ until their stated boundary is met. 11. **Native Host and shared encode.** Revisit Windows first only after capture, hardware encode, audio, RTP feedback, resources, packaging, licensing, and Browser interoperability are proved. -12. **Platform output.** Revisit AirPlay/Cast only when a target Browser and - physical receiver prove the live `MediaStream` contract. +12. **Platform output.** System/tab mirroring needs no product adapter, while + Remote Playback, default Cast and AirPlay do not provide a portable live + `MediaStream` output contract. Reopen only for a registered custom receiver + acting as an ordinary Viewer after a named Browser and physical receiver + pass the [platform-output gate](./research/platform-output.md). 13. **NAT inference and predictive candidates.** Keep multi-STUN observations diagnostic until controlled Browser and target-network evidence proves a connection-scoped classifier with bounded false rejection. Do not hard-skip Peer candidates from a participant-wide NAT label or inject predicted ports - into production ICE without a new accepted decision and a measured gate. + into production ICE without a new accepted decision and a measured gate. See + [Browser NAT traversal](./research/nat-traversal.md). 14. **Unresolved route-state ownership claims.** Reopen lower-revision reauthentication, active SFU failure during an unrelated prepare, and multi-child relay-evidence ownership only from an exact current-wire