AAA
6cc6863294
## Problem and outcome / 问题与结果 Windows native H.264 startup trusted the first enumerated hardware encoder. A failing GPU/driver could end sharing even when another encoder worked. The shared capture/relay selector now tries hardware candidates within its existing startup budget, verifies actual H.264 output, and retains the selected backend for later source/quality changes. Explicit H.264 never silently becomes VP8. A browser ICE candidate rejected with `OperationError` also aborted the remaining queue and could prevent the Viewer from answering. The shared candidate receiver now records that rejection and continues; description/lifecycle errors still propagate, and transport state and the existing route deadline retain failure authority. The launcher no longer prefills the Demo address. Site mode and its explicit origin authorization remain unchanged. ## Release notes ### 中文 这是一次兼容性修复更新，改善 Windows App 的 H.264 分享启动和浏览器的媒体连接建立。 - 首选硬件编码器无法工作时，尝试其他可用的 H.264 硬件编码器；手动选择 H.264 时仍保持该编码格式。 - 修复单条被拒绝的 ICE 候选中断后续候选处理、导致连接建立失败的问题。 - App 的已有站点地址不再预填 Demo；已保存的地址保持不变。 协议和配置格式不变，可直接升级。缺少双方诊断报告的现场连接失败仍待核实。 ### English This maintenance update improves H.264 sharing startup in the Windows App and media connection setup in browsers. - Try other available H.264 hardware encoders when the preferred encoder cannot work. Manually selected H.264 remains H.264. - Fix a rejected ICE candidate interrupting subsequent candidate processing and preventing connection setup. - Stop prefilling the Demo address in App Site mode while preserving saved addresses. Protocol and configuration formats are unchanged; no migration is required. Reported connection failures without matched diagnostics remain unconfirmed. ## Verification / 验证 - `npm run check`: 74 test files / 1,057 tests, type-check and Web build passed. - Documentation and repository hygiene checks passed. - `npm run check:go`: formatting, Go tests/vet, Windows/Linux core builds and Windows native regressions/compile passed. Darwin cgo packaging belongs to the native macOS CI runner. - Real hardware: a preferred AMD encoder failed activation; the selector reached a working NVIDIA H.264 encoder without changing codec. Local native Host gate passed decoded Viewer frames, live quality changes, source replacement and cleanup. - Browser Host/Viewer queue regressions fail before the ICE repair and pass afterward. Chrome 152 rejected an injected mismatched media identifier, then connected using valid candidates. - External-host public invitation HTTP/WebSocket gate and native H.264 RTP delivery gate passed. The remote Pion gate establishes RTP receipt, not physical remote decoding or endurance. ## Impact / 影响 Patch release v1.6.1; no wire, persistence, route-policy or dependency changes. Browser/App/Server public compatibility is preserved. Shared media/routing owners and scoped research/TODO are updated; unknown reporter causes remain explicit. No claim of universal H.264 driver recovery, improved NAT success on every network, or a fix for provider HTTP 1033. Site-mode redesign and 4K remain outside this phase.
Windows Capture Process
This isolated Windows process provides native capture for Piik App. It enumerates local displays and visible top-level windows, binds each window to its PID and process creation time, and has independent modes for:
- capability discovery (
--probe); - process-tree or default-device loopback PCM (
--capture-audio); or - WGC/D3D11 screen/window video with adapter-bound Media Foundation H.264 or
libvpx VP8 output (
--capture-video --codec auto|h264|vp8). - bounded 320x180 BMP source previews (
--preview), delivered once per target over the local control connection rather than the media route.
The video command receives the current product resolution, frame-rate, bitrate, and quality preference. A replacement process applies live changes while the Go session retains its Pion source and connections.
Sharing and source previews request borderless access by default when Windows
exposes IsBorderRequired and GraphicsCaptureAccess (build 20348+, including
Windows 11). The probe advertises API availability; the source-picker preference
passes --show-capture-border when the Host wants the sharing border visible.
One capture-local owner applies completed access without waiting
for consent, and cancels pending access on retirement. Unsupported APIs, denial
or failure retain ordinary capture; another capture session may still require a
border. The property value is a preference, not proof that the border is invisible.
Consumer Windows 10 through build 19045 lacks this API; the current WGC capture
path keeps its system border there.
Video and audio run as separate bounded child processes. A source whose audio loopback cannot be initialized keeps video available and reports audio unavailable instead of failing the whole source. Process loopback is probed by activation rather than inferred from a Windows build number; display sources use the standard render-device loopback available on Windows 10 and later.
Auto compares target-profile encoding work within a four-second selection
budget; H.264 that meets the target needs no software comparison. The selected
codec remains fixed across profile and source changes. VP8 reads the existing
NV12 surface through one staging texture and uses the same encoded-frame
boundary. The process has no network fallback. The App
consumes the selected process or system-audio stream through its native media
edge when the capability probe reports support. Build to the stable project
build/go-check directory for a bounded capability run:
npm run check:native
The checks include synthetic output-worker replacement and failure cases using WARP and a test codec, plus preview cleanup when WGC rejects a hidden test window. They do not capture a user screen or validate a physical encoder.
The implementation follows Microsoft's MIT-licensed reference samples and
official API contracts without copying their WIL framework. The retained MF
fixture compiles the same encoder source with PIIK_H264_FIXTURE; there is
not a second product MFT implementation.
Shared local outputs use stock WebRTC source/encoder adaptation. H264 keeps
native NV12 surfaces, while WebRTC VP8 reads back into I420. The existing capture
and output mailboxes retain source/generation ownership; adapted sizes and
timestamps cross the same media envelope. Hardware low-latency mode stays on;
the old fixed one-frame VBV override is removed so later bitrate increases are
not trapped by the original buffer limit.
The build caches the pinned WebRTC SDK and matching official MSVC linker/runtime
libraries under build/encoder-pool. The SDK supplies libvpx, replacing the
separate source build. Only statically linked code and required notices enter
the package, not the SDK/toolchain or another runtime service.
- https://github.com/microsoft/Windows-classic-samples/tree/main/Samples/ApplicationLoopback
- https://github.com/microsoft/Windows.UI.Composition-Win32-Samples/tree/master/cpp/ScreenCaptureforHWND
- https://learn.microsoft.com/en-us/windows/win32/api/audioclientactivationparams/ns-audioclientactivationparams-audioclient_activation_params
- https://learn.microsoft.com/en-us/windows/uwp/audio-video-camera/screen-capture
- https://learn.microsoft.com/en-us/windows/win32/api/windows.graphics.capture.interop/nf-windows-graphics-capture-interop-igraphicscaptureiteminterop-createforwindow
- https://learn.microsoft.com/en-us/uwp/api/windows.graphics.capture.direct3d11captureframepool.createfreethreaded
- https://learn.microsoft.com/en-us/uwp/api/windows.graphics.capture.graphicscapturesession.isborderrequired
- https://github.com/microsoft/Windows.UI.Composition-Win32-Samples/issues/97#issuecomment-1015962518