Files
Piik/docs/research/native-sender.md
T
AAA 1332473513 feat: rename product and runtime to Piik
Rename the active product, Go module, binaries, configuration, protocols and documentation to Piik while retaining the TV mascot and existing media/room behavior.

Use one current private contract without old-name aliases. Preserve SQLite authority and copy compatible local settings once; other settings start fresh. Git-message cleanup and the infrastructure name cutover are coordinated with this phase, with original history and deployment data archived externally.

Validated locally with Web and Go checks, Windows capture compilation, matched Client/Server builds, packaged Client control handshake and history-rewrite integrity checks. Skip duplicate hosted CI for this owner-authorized local release.

[skip ci]
2026-09-10 11:10:12 +08:00

8.2 KiB

Native Sender And Shared-Encode Evidence

  • Reviewed: 2026-08-27
  • Scope: Windows capture, one hardware H.264 encoder, bounded WebRTC fanout
  • Status: historical evidence; obsolete sender implementation deleted

Current product scope is owned by media quality. ADR-0006 owns the stop line.

Conclusion

A native sender can plausibly capture and encode once while serving bounded independent WebRTC transports. Shared encoding can reduce capture/encode work; it cannot remove each child's RTP, congestion, retransmission, encryption, and network copy.

The retained measurements prove:

  • Pion can fan one encoded source into two independent transport bindings;
  • one WebCodecs encoder object fed two Browser sessions in a bounded fixture;
  • one Windows Graphics Capture/Media Foundation H.264 path used an adapter- bound hardware MFT with matching process/LUID VideoEncode activity; and
  • the product helper delivered H.264 video and process-tree audio to one Chrome Viewer.

It does not prove two-Viewer product fanout, heterogeneous congestion feedback, loss/retransmission correctness, audio synchronization, reconnect isolation, public networks, endurance, packaging, update/signing, cross-platform behavior, or distribution licensing. Native remains outside the current release.

Candidate Shape

The smallest measured Windows pipeline was:

opaque window target
  -> Windows Graphics Capture BGRA surface
  -> one D3D11 conversion to NV12
  -> one adapter-bound hardware Media Foundation H.264 encoder
  -> one generation-bound Go/Pion RTP source
  -> at most two independent PeerConnections

Each PeerConnection must keep independent ICE, DTLS-SRTP, SSRC/sequence/TWCC, pacing, RTCP, congestion, and retransmission state. PLI/FIR from any child may be coalesced into one shared SPS/PPS/IDR request, but a weak child's bitrate request cannot silently become a room-wide minimum policy. A third Viewer must wait under the same endpoint capacity as Web Hosts.

Local target identity, HWND/PID, process creation time, adapter/MFT identity, PCM, and hardware diagnostics never enter signaling. The helper revalidates the selected window/process before capture. There is no software, codec, monitor, or audio fallback hidden behind a hardware-only source choice.

Browser And Pion Fixtures

A stack of bounded fixtures established progressively narrower facts:

Gate Retained result Missing proof
One Pion RTP source, two bindings Equivalent payload reached two transport-local SSRCs. No encoder or Browser.
WebCodecs VP8, two Chrome receivers One encoder object produced 120 outputs; both decoded. Physical/hardware encoder identity.
12-second live bridge 360 inputs/outputs; both Viewers rendered 331 frames; queues remained bounded. Product rooms/capture/audio.
Pion GCC plus negotiated RTX Failed with unknown RTX SSRC. Valid shared congestion/retransmission composition.
No-RTX replay One leg recovered without disturbing the other. Accurate retransmission-specific loss accounting.
Two-leg stock GCC Both estimates happened to be 600 kbps; one shared reconfiguration and keyframe recovery succeeded. Heterogeneous estimates and product policy.

The stock-GCC/RTX composition remains no-go. Do not replace it with a custom pacer, congestion controller, interceptor fork, SRTP, or private transport. No-RTX replay remains research-only because it can distort RTP/RTCP loss accounting.

Hardware Encoder Evidence

WebCodecs

Chrome accepted H.264 Annex-B 1280x720@30 with prefer-hardware and produced 360/360 frames, but Browser-process GPU samples showed no attributed VideoEncode activity. The emitted SPS was 42041f, not the requested codec string. This proves usable chunks and in-band recovery units, not hardware selection or exact WebRTC fmtp compatibility. VP8 prefer-hardware was rejected on the same Windows system.

Media Foundation Fixture

One MSVC/D3D11 fixture selected the NVIDIA H.264 Encoder MFT on an RTX 4070 SUPER by exact DXGI LUID, required hardware enumeration, D3D11 awareness, low-latency mode, CBR readback, 3 Mbps, a 60-frame GOP, and forced keyframes.

Measure Retained result
Inputs / outputs 360 / 360
Recovery units 6 / 6 with Annex-B SPS, PPS, and IDR
Maximum in flight 1
Wall time 12.106 s
Input-to-output p95 11.575 ms
Process+adapter VideoEncode 45 samples; mean 2.949%, max 4.863%
Emitted profile 42c01f

The MFT was physically attributed by hardware-only activation, accepted device manager, exact process/LUID GPU engine, and output progress. Its 42c01f Constrained Baseline profile was absent from Pion's default exact mode-1 list, so the fixture correctly stopped before claiming transport interoperability. The later opt-in fixture registered that exact profile and proved one Browser loopback; it did not alter the wider stop line.

Product Helper Smoke

The Windows helper reuses the same encoder source with WGC capture and process-tree WASAPI loopback. One bounded run delivered 203 rendered 1280x720 frames and 500 Opus packets to a Chrome Viewer. The exact helper PID and adapter LUID matched nonzero VideoEncode. This is a one-Viewer functional proof, not a resource or product release claim.

Remaining Gates

Before native product consideration, one current-wire candidate must prove:

  1. current room/Host authorization and source-generation cleanup;
  2. exact emitted H.264 profile negotiation and unmodified Chrome, Edge, Firefox, and Safari decode/render;
  3. two independent transports with one physical encoder and compatible source, codec, cadence, and rate requests;
  4. correct PLI/FIR, NACK/RTX, TWCC, burst/sustained loss, and reconnect isolation;
  5. process audio, A/V synchronization, source switch, pause, and failure;
  6. bounded CPU/GPU, memory, latency, quality, upload, queues, and thermals under real games and public networks;
  7. evaluation packaging followed by installer, signing, update, and recovery; and
  8. project, dependency, H.264 patent, and distribution license decisions.

No native experiment may preserve an obsolete Piik wire, create another room/topology model, loosen endpoint capacity, or become production merely to make a diagnostic gate pass.

License Boundary

Pion is MIT and coder/websocket is ISC. The Windows APIs use installed hardware MFTs; vendor codec binaries are not redistributed. Direct NVENC would require a separate NVIDIA Video Codec SDK and redistribution review. H.264 patent and project licensing remain unresolved before executable distribution. No proprietary Oopz/Agora or GPL/AGPL code was copied.

Primary Sources