fix(media): recover H264 selection and isolate rejected ICE candidates
## Problem and outcome / 问题与结果 Windows native H.264 startup trusted the first enumerated hardware encoder. A failing GPU/driver could end sharing even when another encoder worked. The shared capture/relay selector now tries hardware candidates within its existing startup budget, verifies actual H.264 output, and retains the selected backend for later source/quality changes. Explicit H.264 never silently becomes VP8. A browser ICE candidate rejected with `OperationError` also aborted the remaining queue and could prevent the Viewer from answering. The shared candidate receiver now records that rejection and continues; description/lifecycle errors still propagate, and transport state and the existing route deadline retain failure authority. The launcher no longer prefills the Demo address. Site mode and its explicit origin authorization remain unchanged. ## Release notes ### 中文 这是一次兼容性修复更新,改善 Windows App 的 H.264 分享启动和浏览器的媒体连接建立。 - 首选硬件编码器无法工作时,尝试其他可用的 H.264 硬件编码器;手动选择 H.264 时仍保持该编码格式。 - 修复单条被拒绝的 ICE 候选中断后续候选处理、导致连接建立失败的问题。 - App 的已有站点地址不再预填 Demo;已保存的地址保持不变。 协议和配置格式不变,可直接升级。缺少双方诊断报告的现场连接失败仍待核实。 ### English This maintenance update improves H.264 sharing startup in the Windows App and media connection setup in browsers. - Try other available H.264 hardware encoders when the preferred encoder cannot work. Manually selected H.264 remains H.264. - Fix a rejected ICE candidate interrupting subsequent candidate processing and preventing connection setup. - Stop prefilling the Demo address in App Site mode while preserving saved addresses. Protocol and configuration formats are unchanged; no migration is required. Reported connection failures without matched diagnostics remain unconfirmed. ## Verification / 验证 - `npm run check`: 74 test files / 1,057 tests, type-check and Web build passed. - Documentation and repository hygiene checks passed. - `npm run check:go`: formatting, Go tests/vet, Windows/Linux core builds and Windows native regressions/compile passed. Darwin cgo packaging belongs to the native macOS CI runner. - Real hardware: a preferred AMD encoder failed activation; the selector reached a working NVIDIA H.264 encoder without changing codec. Local native Host gate passed decoded Viewer frames, live quality changes, source replacement and cleanup. - Browser Host/Viewer queue regressions fail before the ICE repair and pass afterward. Chrome 152 rejected an injected mismatched media identifier, then connected using valid candidates. - External-host public invitation HTTP/WebSocket gate and native H.264 RTP delivery gate passed. The remote Pion gate establishes RTP receipt, not physical remote decoding or endurance. ## Impact / 影响 Patch release v1.6.1; no wire, persistence, route-policy or dependency changes. Browser/App/Server public compatibility is preserved. Shared media/routing owners and scoped research/TODO are updated; unknown reporter causes remain explicit. No claim of universal H.264 driver recovery, improved NAT success on every network, or a fix for provider HTTP 1033. Site-mode redesign and 4K remain outside this phase.
This commit is contained in:
18 files changed
+316
-54
No files matched your search
@@ -170,14 +170,22 @@ DNS failures, API rejection and networks blocking both transports can still fail
|
||||
A 2026-09-21 Windows check used the packaged binary with an isolated UDP sink and
|
||||
TCP forwarder. The previous arguments timed out at 30 seconds without trying TCP.
|
||||
The candidate connected in about 11 seconds, served the public page and echoed a
|
||||
WebSocket message, then joined the child on close. A separate ordinary-network
|
||||
run registered in about five seconds but this host's public HTTPS/WSS requests
|
||||
ended with EOF; end-to-end access on that path remains unverified. These checks
|
||||
do not establish the cause of reports without matching diagnostics. Upstream
|
||||
WebSocket message, then joined the child on close. The ordinary-network App link
|
||||
gate also serves the invitation page and upgrades its WebSocket from an external
|
||||
US host, then confirms App exit and link retirement. These bounded checks do not
|
||||
establish regional reachability, uninterrupted connector availability or the
|
||||
cause of reports without matching diagnostics. Upstream
|
||||
[Quick Tunnel setup](https://github.com/cloudflare/cloudflared/blob/2026.8.3/cmd/cloudflared/tunnel/quick_tunnel.go)
|
||||
and [protocol fallback](https://github.com/cloudflare/cloudflared/blob/2026.8.3/supervisor/tunnel.go)
|
||||
own the dependency behavior; Piik adds no separate retry loop.
|
||||
|
||||
[HTTP 1033](https://developers.cloudflare.com/support/troubleshooting/http-status-codes/cloudflare-1xxx-errors/error-1033/)
|
||||
means Cloudflare cannot find a healthy tunnel connector. App readiness requires
|
||||
both the assigned origin and a registered connection; later reconnection belongs
|
||||
to cloudflared, while process exit ends that App's public-link runtime. This
|
||||
HTTP failure precedes WebRTC candidate exchange and is distinct from media-route
|
||||
exhaustion on an already reachable site.
|
||||
|
||||
## Remaining Gates
|
||||
|
||||
- Run the exact clean-revision package on macOS; Linux amd64 is proved.
|
||||
|
||||
@@ -87,6 +87,15 @@ An earlier remote description is not readiness for a new generation's candidates
|
||||
Native event delivery can precede its offer/answer response; the existing
|
||||
negotiation queue retains those candidates until the matching answer is applied.
|
||||
|
||||
The Browser queue must also isolate a candidate-processing `OperationError`.
|
||||
Fault injection showed that one rejection formerly skipped later candidates in
|
||||
both Host and Viewer queues and could prevent the Viewer's SDP answer. Chrome
|
||||
152 independently rejected a mismatched media identifier while the connection
|
||||
still succeeded with valid candidates. The shared receiver now records that
|
||||
individual rejection and continues; missing-description errors are not hidden.
|
||||
This agrees with [WebRTC candidate processing](https://www.w3.org/TR/webrtc/#dom-peerconnection-addicecandidate)
|
||||
and does not establish the cause of undiagnosed field route failures.
|
||||
|
||||
### Native Shared-Socket Preflight
|
||||
|
||||
Native requests one dual-stack wildcard UDP socket through Go and Pion's existing
|
||||
|
||||
@@ -148,6 +148,10 @@ The retained hardware fixture also changed one live NVIDIA MFT through
|
||||
produced about `1.00 MB -> 0.68 MB -> 0.97 MB`, while ordered 30 fps output and
|
||||
the pinned profile remained intact. Both AMD MFT candidates failed activation
|
||||
before this probe, so the evidence is not a cross-vendor dynamic-rate contract.
|
||||
With the failing AMD candidate preferred, bounded product selection now reaches
|
||||
the working NVIDIA H264 encoder without changing codec. A local native Host gate
|
||||
also passes decoded Viewer output, quality changes and source replacement. This
|
||||
establishes the selection defect and repair, not the unknown reporter's cause.
|
||||
The product therefore observes GCC for routing but does not yet apply one edge's
|
||||
target globally to the shared encoder.
|
||||
|
||||
|
||||
@@ -121,7 +121,13 @@ does not by itself prove hardware acceleration, and Web content cannot select a
|
||||
specific MFT, NVENC, AMF, or QSV implementation.
|
||||
|
||||
Windows native capture uses the same VP8/Auto/H264 controls. H264 selects the
|
||||
hardware path and VP8 the bundled libvpx encoder. Auto measures encoding work
|
||||
hardware path and VP8 the bundled libvpx encoder. Hardware enumeration is only
|
||||
a candidate list: startup tries the preferred encoder first, then remaining
|
||||
adapter/encoder candidates within the shared selection budget. Manual H264
|
||||
requires actual H264 output and never changes codec on failure. The selected
|
||||
device owns capture and output workers; its identity is retained as the first
|
||||
choice for quality/source replacement. Native relay encoding uses the same
|
||||
selection owner. Auto measures encoding work
|
||||
for synthetic NV12 frames at the selected dimensions and frame rate, with a
|
||||
bounded warmup and sample. If H264 sustains the target it is selected; otherwise
|
||||
VP8 is measured within the remaining four-second budget. This is a throughput
|
||||
|
||||
@@ -139,6 +139,12 @@ identity still match; future revisions and retired connections are rejected.
|
||||
Preparation reports retain their exact operation fence. Reports without a
|
||||
connection identity retain their exact room-revision fence.
|
||||
|
||||
An individually rejected ICE candidate is disposable input. Browser
|
||||
`addIceCandidate` candidate-processing errors are recorded without aborting the
|
||||
remaining queue or SDP answer; missing-description/lifecycle errors remain
|
||||
visible. Native input follows the same boundary. Actual transport state and the
|
||||
route operation's deadline own failure, not one rejected candidate.
|
||||
|
||||
## SFU Fallback
|
||||
|
||||
The only application fallback is one admitted embedded SFU publication from the
|
||||
|
||||
+17
-1
@@ -1,12 +1,17 @@
|
||||
# Current TODO Ledger
|
||||
|
||||
Last reviewed: 2026-09-21
|
||||
Last reviewed: 2026-09-22
|
||||
|
||||
Only **Now** is executable. Product modules own behavior; Git/PRs own completed
|
||||
history. A parked idea is not implementation authority.
|
||||
|
||||
## Now
|
||||
|
||||
- [ ] **Passive App attachment: design hold.** Site mode authorizes one selected
|
||||
origin and supplies native media without starting a local room server. A
|
||||
passive replacement needs an accepted site-consent/discovery flow; it must not
|
||||
admit arbitrary sites or add another runtime owner. Keep Site mode until that
|
||||
decision; removal of the Demo prefill does not authorize changing site trust.
|
||||
- [ ] **Complete manual accessible-name review.** Verify the empty video's
|
||||
screen-reader output: Chromium exposes an unavailable-media
|
||||
description despite the literal shared-picture label and no media error. Also
|
||||
@@ -26,6 +31,12 @@ history. A parked idea is not implementation authority.
|
||||
|
||||
## Awaiting Device Or Reporter Evidence
|
||||
|
||||
- [ ] **No available media route.** Obtain matched Host/Viewer diagnostics from
|
||||
a failed attempt, with version and mode. Inspect candidate exchange, selected
|
||||
paths, first-frame admission and route rejection separately. The Browser
|
||||
candidate-queue rejection defect is repaired and regression-tested; current
|
||||
STUN reachability and route-lifecycle checks do not establish these reporters'
|
||||
causes. HTTP 1033 belongs to the public-link item below, before media routing.
|
||||
- [ ] **Camera and Host microphone device coverage.** The owner accepted the
|
||||
sharing layout and authorized release with these physical limits recorded.
|
||||
Verify real audio levels/echo, multiple-device replacement and native mixing on
|
||||
@@ -41,6 +52,9 @@ history. A parked idea is not implementation authority.
|
||||
bounded protocol fallback. [Runtime evidence](./research/cross-platform-client-runtime.md#public-invitation-startup)
|
||||
separates this fix from DNS, provider and remote access failures. The reporting
|
||||
environments still need version and diagnostic evidence before assigning a cause.
|
||||
Demo access failures and public-link HTTP 1033 also remain unconfirmed: current
|
||||
Demo health/assets and an external-host public-link HTTP/WebSocket gate pass.
|
||||
Check connector reachability separately from WebRTC media availability.
|
||||
- [ ] **Interruption during established viewing.** A Viewer reportedly returns
|
||||
to P2P connecting after watching for a while. Native receiver renegotiation,
|
||||
retired event delivery and SFU replacement have locally reproduced defects
|
||||
@@ -67,6 +81,8 @@ history. A parked idea is not implementation authority.
|
||||
paired Debug reports are still unavailable. Distinguish site authorization,
|
||||
browser permission, control capacity, enumeration and capture-start failures;
|
||||
local permission and recovery checks do not establish the reporters' causes.
|
||||
Manual H264 returning immediately to idle also needs reporter diagnostics;
|
||||
bounded selection past an unusable hardware encoder is locally verified.
|
||||
- [ ] **Windows 32-bit candidate acceptance.** Verify the isolated
|
||||
`spike/windows-x86-capture` candidate's launch, capture/audio, memory pressure,
|
||||
source replacement and update links on a 32-bit Windows device. WOW64
|
||||
|
||||
@@ -16,6 +16,23 @@ import (
|
||||
"github.com/pion/webrtc/v4"
|
||||
)
|
||||
|
||||
func writeCaptureFixtureFrame(frame nativecapture.Frame) {
|
||||
var header [32]byte
|
||||
copy(header[:], "SMED")
|
||||
header[4], header[5], header[7] = 2, byte(frame.Kind), byte(frame.Layer)
|
||||
if frame.KeyFrame {
|
||||
header[6] = 1
|
||||
}
|
||||
binary.BigEndian.PutUint64(header[8:16], uint64(frame.Timestamp/(100*time.Nanosecond)))
|
||||
binary.BigEndian.PutUint64(header[16:24], uint64(frame.Duration/(100*time.Nanosecond)))
|
||||
binary.BigEndian.PutUint16(header[24:26], uint16(frame.Width))
|
||||
binary.BigEndian.PutUint16(header[26:28], uint16(frame.Height))
|
||||
binary.BigEndian.PutUint32(header[28:32], uint32(len(frame.Data)))
|
||||
if _, err := os.Stdout.Write(append(header[:], frame.Data...)); err != nil {
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
func audioRecoveryProfile() nativecapture.VideoProfile {
|
||||
return nativecapture.VideoProfile{Width: 1280, Height: 720, Framerate: 30,
|
||||
Bitrate: 2_000_000, Preference: "balanced"}
|
||||
@@ -26,22 +43,7 @@ func audioRecoveryProfile() nativecapture.VideoProfile {
|
||||
func runAudioRecoveryCapture() {
|
||||
stopped := make(chan struct{})
|
||||
go func() { _, _ = io.Copy(io.Discard, os.Stdin); close(stopped) }()
|
||||
write := func(frame nativecapture.Frame) {
|
||||
var header [32]byte
|
||||
copy(header[:], "SMED")
|
||||
header[4], header[5], header[7] = 2, byte(frame.Kind), byte(frame.Layer)
|
||||
if frame.KeyFrame {
|
||||
header[6] = 1
|
||||
}
|
||||
binary.BigEndian.PutUint64(header[8:16], uint64(frame.Timestamp/(100*time.Nanosecond)))
|
||||
binary.BigEndian.PutUint64(header[16:24], uint64(frame.Duration/(100*time.Nanosecond)))
|
||||
binary.BigEndian.PutUint16(header[24:26], uint16(frame.Width))
|
||||
binary.BigEndian.PutUint16(header[26:28], uint16(frame.Height))
|
||||
binary.BigEndian.PutUint32(header[28:32], uint32(len(frame.Data)))
|
||||
if _, err := os.Stdout.Write(append(header[:], frame.Data...)); err != nil {
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
write := writeCaptureFixtureFrame
|
||||
microphone := len(os.Args) > 1 && os.Args[1] == "--capture-microphone"
|
||||
if microphone {
|
||||
if len(os.Args) == 4 && os.Args[2] == "--device" && os.Args[3] == "missing" {
|
||||
|
||||
@@ -408,6 +408,8 @@ func (session *Session) ReplaceSource(
|
||||
}
|
||||
profile := session.profile
|
||||
options.Codec = session.videoOptions.Codec
|
||||
options.AdapterIndex = session.videoOptions.AdapterIndex
|
||||
options.EncoderIndex = session.videoOptions.EncoderIndex
|
||||
options.OutputGroups = session.edgeCapacity
|
||||
hasAudio := session.audioSource != nil && session.audioStream != nil
|
||||
session.mu.Unlock()
|
||||
@@ -506,6 +508,7 @@ func (session *Session) commitCapture(
|
||||
if state.RestoreToken != "" {
|
||||
options.RestoreToken = state.RestoreToken
|
||||
}
|
||||
state.applyBackend(&options)
|
||||
session.stream = replacement
|
||||
applied := make(chan struct{})
|
||||
session.captureApplied = applied
|
||||
@@ -754,6 +757,7 @@ func (session *Session) runVideo() error {
|
||||
session.videoOptions.RestoreToken = status.RestoreToken
|
||||
}
|
||||
}
|
||||
status.applyBackend(&session.videoOptions)
|
||||
session.mu.Unlock()
|
||||
slog.Debug("piik-client", "event", "capture-state", "state", status.State, "codec", status.Codec,
|
||||
"share", diagnostics.ID(session.shareID), "width", status.Width, "height", status.Height, "fps", status.FPS,
|
||||
@@ -910,6 +914,7 @@ func waitForCaptureProfile(
|
||||
}
|
||||
}
|
||||
inputSeen := false
|
||||
var starting CaptureState
|
||||
for {
|
||||
frame, err := stream.Read()
|
||||
if err != nil {
|
||||
@@ -952,9 +957,11 @@ func waitForCaptureProfile(
|
||||
send(result{stage: "profile-mismatch", err: errors.New("native capture profile was not applied")})
|
||||
return
|
||||
}
|
||||
state.AdapterIndex, state.EncoderIndex = starting.AdapterIndex, starting.EncoderIndex
|
||||
send(result{state: state})
|
||||
return
|
||||
}
|
||||
starting = state
|
||||
if !send(result{state: state}) {
|
||||
return
|
||||
}
|
||||
@@ -1129,6 +1136,17 @@ func decodeCaptureState(payload []byte) (CaptureState, error) {
|
||||
return state, nil
|
||||
}
|
||||
|
||||
// Selected hardware is a capture fact, not the page's enumeration preference.
|
||||
// Keep it for quality/source replacement without changing the wire contract.
|
||||
func (state CaptureState) applyBackend(options *nativecapture.VideoOptions) {
|
||||
if state.AdapterIndex != nil {
|
||||
options.AdapterIndex = *state.AdapterIndex
|
||||
}
|
||||
if state.EncoderIndex != nil {
|
||||
options.EncoderIndex = *state.EncoderIndex
|
||||
}
|
||||
}
|
||||
|
||||
func validH264ProfileLevelID(value string) bool {
|
||||
switch value {
|
||||
case "42c01e", "42c01f", "42c020", "42c028", "42c029", "42c02a", "42c032", "42c033":
|
||||
|
||||
@@ -17,6 +17,13 @@ import (
|
||||
)
|
||||
|
||||
func TestMain(tests *testing.M) {
|
||||
if os.Getenv("PIIK_NATIVEHOST_PIPE_FIXTURE") == "backend" {
|
||||
for _, key := range []string{"PIIK_CAPTURE_STARTING", "PIIK_CAPTURE_ACTIVE"} {
|
||||
writeCaptureFixtureFrame(nativecapture.Frame{Kind: nativecapture.FrameStatus, Data: []byte(os.Getenv(key))})
|
||||
}
|
||||
_, _ = io.Copy(io.Discard, os.Stdin)
|
||||
os.Exit(0)
|
||||
}
|
||||
if os.Getenv("PIIK_NATIVEHOST_PIPE_FIXTURE") == "audio-recovery" {
|
||||
runAudioRecoveryCapture()
|
||||
os.Exit(0)
|
||||
@@ -72,9 +79,11 @@ func TestCaptureCommitWaitsForReaderMetadataOrTermination(t *testing.T) {
|
||||
check(err)
|
||||
defer replacement.Close()
|
||||
committed := make(chan error, 1)
|
||||
adapter, encoder := uint32(1), uint32(2)
|
||||
go func() {
|
||||
session.updateMu.Lock()
|
||||
err := session.commitCapture(options, QualityProfile{Video: options.Profile, AudioBitrate: 64_000}, replacement, CaptureState{}, nil, false)
|
||||
err := session.commitCapture(options, QualityProfile{Video: options.Profile, AudioBitrate: 64_000}, replacement,
|
||||
CaptureState{AdapterIndex: &adapter, EncoderIndex: &encoder}, nil, false)
|
||||
session.updateMu.Unlock()
|
||||
committed <- err
|
||||
}()
|
||||
@@ -101,6 +110,9 @@ func TestCaptureCommitWaitsForReaderMetadataOrTermination(t *testing.T) {
|
||||
t.Fatal("installed metadata did not acknowledge update")
|
||||
}
|
||||
media := publication.Media()
|
||||
if session.videoOptions.AdapterIndex != adapter || session.videoOptions.EncoderIndex != encoder {
|
||||
t.Fatal("capture replacement discarded its selected hardware backend")
|
||||
}
|
||||
if media.Layers[1].Width != 854 || media.Layers[1].Height != 480 || media.Layers[1].Bitrate != 2_000_000 {
|
||||
t.Fatalf("acknowledged stale metadata: %+v", media)
|
||||
}
|
||||
@@ -142,6 +154,33 @@ func TestCaptureCommitWaitsForReaderMetadataOrTermination(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestCaptureProfileRetainsSelectedBackendFromStarting(t *testing.T) {
|
||||
t.Setenv("PIIK_NATIVEHOST_PIPE_FIXTURE", "backend")
|
||||
t.Setenv("PIIK_CAPTURE_STARTING", string(captureStatePayload(t,
|
||||
`{"state":"starting","hardwareOnly":true,"codec":"h264","adapterIndex":1,"adapterName":"GPU","adapterIdentity":"0:1","encoderIndex":2,"encoderName":"H264","encoderIdentity":"encoder"}`)))
|
||||
t.Setenv("PIIK_CAPTURE_ACTIVE", string(captureStatePayload(t,
|
||||
`{"state":"active","hardwareOnly":true,"codec":"h264","profileLevelId":"42c01f","width":1280,"height":720,"fps":30}`)))
|
||||
executable, err := os.Executable()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
profile := nativecapture.VideoProfile{Width: 1280, Height: 720, Framerate: 30, Bitrate: 3_000_000, Preference: "balanced"}
|
||||
stream, err := nativecapture.StartVideo(t.Context(), executable, nativecapture.VideoOptions{
|
||||
Target: nativecapture.CaptureTarget{Kind: "display", SourceID: "1", Title: "Fixture"}, Codec: "h264", Profile: profile, OutputGroups: 4,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer stream.Close()
|
||||
state, err := waitForCaptureProfile(t.Context(), stream, profile, "h264", false, false)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if state.AdapterIndex == nil || *state.AdapterIndex != 1 || state.EncoderIndex == nil || *state.EncoderIndex != 2 {
|
||||
t.Fatalf("active profile lost the actual hardware selection: %+v", state)
|
||||
}
|
||||
}
|
||||
|
||||
func TestOutputPlanPreservesOriginalStartupAndIndependentActivation(t *testing.T) {
|
||||
t.Setenv("PIIK_NATIVEHOST_PIPE_FIXTURE", "echo")
|
||||
executable, err := os.Executable()
|
||||
|
||||
@@ -1265,31 +1265,96 @@ struct VideoEncoderSelection final {
|
||||
std::unique_ptr<VideoEncoder> initial;
|
||||
};
|
||||
|
||||
VideoEncoderSelection SelectVideoEncoder(
|
||||
const ProductArguments& arguments, const Adapter& adapter,
|
||||
const DeviceContext& device) {
|
||||
const auto create = [&]() -> std::unique_ptr<VideoEncoder> {
|
||||
ActivationList activations = EnumerateHardwareEncoders(adapter);
|
||||
auto selected = ActivateTransform(activations, arguments.mft_index, device.manager.Get());
|
||||
return std::make_unique<LiveEncoder>(std::move(selected), arguments.profile);
|
||||
};
|
||||
if (arguments.codec == "vp8") return {};
|
||||
if (arguments.codec == "h264") return {OutputKind::h264, create()};
|
||||
using EncoderCandidate = std::pair<UINT, UINT>;
|
||||
|
||||
void LogEncoderRejection(EncoderCandidate candidate, const GateFailure& error) {
|
||||
std::osyncstream output(std::cerr);
|
||||
output << "result=encoder-candidate-rejected adapter=" << candidate.first
|
||||
<< " encoder=" << candidate.second << " stage=" << error.stage();
|
||||
if (FAILED(error.result())) output << " hresult=0x" << std::hex << error.result();
|
||||
output << '\n';
|
||||
}
|
||||
|
||||
// Only pre-publication setup tries another hardware candidate. A live worker
|
||||
// keeps its selected device/encoder; source replacement owns any later change.
|
||||
template <typename Prepare>
|
||||
EncoderCandidate SelectEncoderCandidate(std::vector<EncoderCandidate> candidates,
|
||||
EncoderCandidate preferred, EncoderClock::time_point deadline, Prepare prepare) {
|
||||
const auto first = std::find(candidates.begin(), candidates.end(), preferred);
|
||||
if (first != candidates.end()) std::rotate(candidates.begin(), first, first + 1);
|
||||
std::exception_ptr failure;
|
||||
for (const auto& candidate : candidates) {
|
||||
RequireEncoderTime(deadline);
|
||||
try {
|
||||
prepare(candidate);
|
||||
RequireEncoderTime(deadline);
|
||||
return candidate;
|
||||
} catch (const GateFailure& error) {
|
||||
LogEncoderRejection(candidate, error);
|
||||
failure = std::current_exception();
|
||||
}
|
||||
}
|
||||
if (failure) std::rethrow_exception(failure);
|
||||
Fail("mft-hardware-enumeration", "no usable hardware H264 encoder was found");
|
||||
}
|
||||
|
||||
VideoEncoderSelection SelectVideoEncoder(
|
||||
ProductArguments& arguments, const std::vector<Adapter>& adapters,
|
||||
DeviceContext& device) {
|
||||
if (arguments.codec == "vp8") {
|
||||
device = CreateDevice(SelectAdapter(adapters, arguments.adapter_index));
|
||||
return {};
|
||||
}
|
||||
const auto began = EncoderClock::now();
|
||||
const auto budget = std::chrono::seconds(4);
|
||||
const auto deadline = began + budget;
|
||||
const auto hardware_deadline = arguments.codec == "auto" ? began + budget / 2 : deadline;
|
||||
std::optional<double> hardware_work;
|
||||
std::unique_ptr<VideoEncoder> hardware;
|
||||
try {
|
||||
auto hardware = create();
|
||||
hardware_work = MeasureEncoderWork(hardware.get(), device.device.Get(),
|
||||
arguments.profile, began + budget / 2);
|
||||
if (*hardware_work <= 1.0 / arguments.profile.frame_rate)
|
||||
std::vector<EncoderCandidate> candidates;
|
||||
for (const auto& adapter : adapters) {
|
||||
RequireEncoderTime(hardware_deadline);
|
||||
try {
|
||||
const auto encoders = EnumerateHardwareEncoders(adapter, false);
|
||||
for (UINT index = 0; index < encoders.count; ++index)
|
||||
candidates.emplace_back(adapter.index, index);
|
||||
} catch (const GateFailure& error) {
|
||||
LogEncoderRejection({adapter.index, 0}, error);
|
||||
}
|
||||
}
|
||||
const auto chosen = SelectEncoderCandidate(candidates,
|
||||
{arguments.adapter_index, arguments.mft_index}, hardware_deadline,
|
||||
[&](EncoderCandidate candidate) {
|
||||
const auto& adapter = SelectAdapter(adapters, candidate.first);
|
||||
auto next_device = CreateDevice(adapter);
|
||||
const auto encoders = EnumerateHardwareEncoders(adapter);
|
||||
auto selected = ActivateTransform(encoders, candidate.second, next_device.manager.Get());
|
||||
auto next = std::make_unique<LiveEncoder>(std::move(selected), arguments.profile);
|
||||
if (arguments.codec == "auto") {
|
||||
hardware_work = MeasureEncoderWork(next.get(), next_device.device.Get(),
|
||||
arguments.profile, hardware_deadline);
|
||||
} else {
|
||||
// Enumeration and activation alone do not prove usable H264 output.
|
||||
const auto texture = CreateSyntheticTexture(next_device.device.Get(), 0, arguments.profile);
|
||||
next->Encode(texture.Get(), 0, true, hardware_deadline);
|
||||
}
|
||||
RequireEncoderTime(hardware_deadline);
|
||||
device = std::move(next_device);
|
||||
hardware = std::move(next);
|
||||
});
|
||||
arguments.adapter_index = chosen.first;
|
||||
arguments.mft_index = chosen.second;
|
||||
if (arguments.codec == "h264" || *hardware_work <= 1.0 / arguments.profile.frame_rate)
|
||||
return {OutputKind::h264, std::move(hardware)};
|
||||
} catch (const std::exception&) {
|
||||
// No media has been published; a failed hardware probe may try software.
|
||||
} catch (const GateFailure&) {
|
||||
if (arguments.codec == "h264") throw;
|
||||
hardware_work.reset();
|
||||
hardware.reset();
|
||||
device = DeviceContext{};
|
||||
}
|
||||
RequireEncoderTime(deadline);
|
||||
if (!device.device) device = CreateDevice(SelectAdapter(adapters, arguments.adapter_index));
|
||||
try {
|
||||
const double software_work = MeasureEncoderWork(nullptr, device.device.Get(),
|
||||
arguments.profile, deadline);
|
||||
@@ -1298,8 +1363,6 @@ VideoEncoderSelection SelectVideoEncoder(
|
||||
if (!hardware_work) throw;
|
||||
}
|
||||
RequireEncoderTime(deadline);
|
||||
auto hardware = create();
|
||||
RequireEncoderTime(deadline);
|
||||
return {OutputKind::h264, std::move(hardware)};
|
||||
}
|
||||
|
||||
@@ -1613,11 +1676,11 @@ ComPtr<ID3D11Texture2D> OwnDecodedTexture(const DeviceContext& device, IMFSample
|
||||
return UploadDecodedNV12(device, pixels.data(), width, height);
|
||||
}
|
||||
|
||||
void RunEncodedVideo(const ProductArguments& arguments) {
|
||||
void RunEncodedVideo(ProductArguments arguments) {
|
||||
auto adapters = EnumerateAdapters();
|
||||
DeviceContext device;
|
||||
auto encoder = SelectVideoEncoder(arguments, adapters, device);
|
||||
const auto& adapter = SelectAdapter(adapters, arguments.adapter_index);
|
||||
const auto device = CreateDevice(adapter);
|
||||
auto encoder = SelectVideoEncoder(arguments, adapter, device);
|
||||
const bool hardware = arguments.codec == "h264";
|
||||
ProtocolWriter writer;
|
||||
WriteVideoStarting(writer, arguments, adapter, encoder);
|
||||
@@ -1694,7 +1757,7 @@ void RunEncodedVideo(const ProductArguments& arguments) {
|
||||
}
|
||||
}
|
||||
|
||||
void RunVideoCapture(const ProductArguments& arguments) {
|
||||
void RunVideoCapture(ProductArguments arguments) {
|
||||
const bool window_target =
|
||||
arguments.target_kind == piik::capture::TargetKind::window;
|
||||
HRESULT identity = window_target
|
||||
@@ -1706,9 +1769,9 @@ void RunVideoCapture(const ProductArguments& arguments) {
|
||||
Check(identity, "target-identity");
|
||||
|
||||
std::vector<Adapter> adapters = EnumerateAdapters();
|
||||
DeviceContext device;
|
||||
auto encoder = SelectVideoEncoder(arguments, adapters, device);
|
||||
const Adapter& adapter = SelectAdapter(adapters, arguments.adapter_index);
|
||||
DeviceContext device = CreateDevice(adapter);
|
||||
auto encoder = SelectVideoEncoder(arguments, adapter, device);
|
||||
const bool hardware = encoder.kind == OutputKind::h264;
|
||||
piik::capture::CapturePresentation presentation(arguments.target_kind,
|
||||
arguments.source_id);
|
||||
|
||||
@@ -114,7 +114,43 @@ void CheckPrimaryFailureDetails() {
|
||||
assert(rejected);
|
||||
}
|
||||
|
||||
void CheckEncoderCandidates() {
|
||||
const std::vector<EncoderCandidate> candidates{{0, 0}, {1, 0}, {1, 1}, {2, 0}};
|
||||
const EncoderCandidate preferred{1, 0};
|
||||
std::vector<EncoderCandidate> tried;
|
||||
const auto deadline = EncoderClock::now() + std::chrono::seconds(1);
|
||||
const auto chosen = SelectEncoderCandidate(candidates, preferred, deadline,
|
||||
[&](EncoderCandidate candidate) {
|
||||
tried.push_back(candidate);
|
||||
if (candidate != EncoderCandidate{1, 1})
|
||||
throw GateFailure("fixture-activation", "candidate is not usable", E_FAIL);
|
||||
});
|
||||
assert((chosen == EncoderCandidate{1, 1}));
|
||||
assert((tried == std::vector<EncoderCandidate>{{1, 0}, {0, 0}, {1, 1}}));
|
||||
tried.clear();
|
||||
try {
|
||||
SelectEncoderCandidate(candidates, chosen, deadline, [&](EncoderCandidate candidate) {
|
||||
tried.push_back(candidate);
|
||||
throw GateFailure("fixture-all-failed", "no working candidate", E_FAIL);
|
||||
});
|
||||
assert(false);
|
||||
} catch (const GateFailure& error) {
|
||||
assert(error.stage() == "fixture-all-failed");
|
||||
}
|
||||
assert(tried.size() == candidates.size() && tried.front() == chosen);
|
||||
tried.clear();
|
||||
try {
|
||||
SelectEncoderCandidate(candidates, preferred, EncoderClock::now(),
|
||||
[&](EncoderCandidate candidate) { tried.push_back(candidate); });
|
||||
assert(false);
|
||||
} catch (const GateFailure& error) {
|
||||
assert(error.stage() == "codec-probe-timeout");
|
||||
}
|
||||
assert(tried.empty());
|
||||
}
|
||||
|
||||
int main() {
|
||||
CheckEncoderCandidates();
|
||||
CheckPrimaryFailureDetails();
|
||||
ComPtr<ID3D11Device> device;
|
||||
Check(D3D11CreateDevice(nullptr, D3D_DRIVER_TYPE_WARP, nullptr, 0, nullptr, 0,
|
||||
|
||||
@@ -7,7 +7,6 @@ import type { HintKind } from "./hints";
|
||||
import { Glyph, type GlyphName } from "../../ui/icons";
|
||||
import { useCopy, type CopyKey } from "../../ui/copy";
|
||||
export type AppMode = "local" | "link" | "site";
|
||||
export const SUGGESTED_SITE = "https://demo.piik.tv";
|
||||
|
||||
const MODES: Array<{
|
||||
mode: AppMode;
|
||||
@@ -170,7 +169,7 @@ export function LauncherForm({
|
||||
value={site}
|
||||
spellCheck={false}
|
||||
inputMode="url"
|
||||
placeholder={SUGGESTED_SITE}
|
||||
placeholder="https://piik.example.com"
|
||||
aria-label={t("client.launch.siteAddress")}
|
||||
onChange={(event) => {
|
||||
onSiteChange(event.target.value);
|
||||
|
||||
@@ -5,7 +5,7 @@ import { BrandMark } from "../components/living/BrandMark";
|
||||
import { LoadingStatus } from "../components/living/WaitingStatus";
|
||||
import { Tooltip } from "../components/living/Tooltip";
|
||||
import { AppHeader } from "../components/living/Header";
|
||||
import { LauncherForm, SUGGESTED_SITE, type AppMode } from "../components/living/LauncherForm";
|
||||
import { LauncherForm, type AppMode } from "../components/living/LauncherForm";
|
||||
import { Btn, Pill } from "../components/living/primitives";
|
||||
import { Glyph } from "../ui/icons";
|
||||
import { hasCopyPreference, useCopy, type CopyKey } from "../ui/copy";
|
||||
@@ -57,7 +57,7 @@ export function AppLauncherPage() {
|
||||
.then((state) => {
|
||||
if (!current) return;
|
||||
setMode(state.defaultMode);
|
||||
setSite(state.site || SUGGESTED_SITE);
|
||||
setSite(state.site);
|
||||
setLocalAccessPassword(state.localAccessPassword);
|
||||
setAppDebug(state.debug);
|
||||
setDebug(state.debug === true || browserDebugEnabled);
|
||||
|
||||
@@ -7,7 +7,7 @@ import { StageTv } from "../components/living/Stage";
|
||||
import { HostMicrophone, HostMicrophoneSettings } from "../components/living/HostMicrophone";
|
||||
import { PlaybackControls } from "../components/living/PlaybackControls";
|
||||
import { SharingSettings } from "../components/living/SharingSettings";
|
||||
import { LauncherForm, SUGGESTED_SITE, type AppMode } from "../components/living/LauncherForm";
|
||||
import { LauncherForm, type AppMode } from "../components/living/LauncherForm";
|
||||
import { QualityPresets } from "../components/living/QualityPresets";
|
||||
import { RoomCodeInput } from "../components/living/RoomCodeInput";
|
||||
import type { QualityProfileId } from "../media/quality";
|
||||
@@ -42,7 +42,7 @@ export function ControlsPreview() {
|
||||
const [microphoneDevice, setMicrophoneDevice] = useState("");
|
||||
const [sharingSettings, setSharingSettings] = useState(false);
|
||||
const [launchMode, setLaunchMode] = useState<AppMode>("link");
|
||||
const [launchSite, setLaunchSite] = useState(SUGGESTED_SITE);
|
||||
const [launchSite, setLaunchSite] = useState("");
|
||||
const [launchPassword, setLaunchPassword] = useState("");
|
||||
const [preset, setPreset] = useState<QualityProfileId>("1080p30");
|
||||
const [policy, setPolicy] = useState<"open" | "private">("open");
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import type { SignalPayload } from "../../shared/protocol";
|
||||
import { candidateSignalOrigin } from "../../shared/nat-candidate";
|
||||
import { debugError } from "../lib/debug";
|
||||
|
||||
export type SignalCandidate = Extract<
|
||||
SignalPayload,
|
||||
@@ -28,11 +29,17 @@ export async function addRemoteIceCandidate(
|
||||
try {
|
||||
await connection.addIceCandidate(candidate);
|
||||
} catch (error) {
|
||||
// A stale/rejected candidate is disposable input, not failed SDP or a dead
|
||||
// transport. Keep draining this generation; ICE state owns route failure.
|
||||
if (
|
||||
!(error instanceof DOMException && error.name === "OperationError") &&
|
||||
candidateSignalOrigin(candidate?.candidate ?? null) !== "predicted"
|
||||
) {
|
||||
throw error;
|
||||
}
|
||||
debugError("webrtc", "remote-candidate-rejected", error, {
|
||||
origin: candidateSignalOrigin(candidate?.candidate ?? null),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1142,6 +1142,28 @@ describe("HostPeer source replacement", () => {
|
||||
peer.dispose();
|
||||
});
|
||||
|
||||
it("drains valid restart candidates after an old-generation candidate is rejected", async () => {
|
||||
const peer = createPeer(createStream(createTrack("video", "video"), null));
|
||||
await peer.start();
|
||||
const connection = FakePeerConnection.latest!;
|
||||
await peer.acceptSignal({ kind: "description", connectionId: peer.connectionId,
|
||||
description: { type: "answer", sdp: "previous-answer" } });
|
||||
await peer.restartIce();
|
||||
const stale = { candidate: "stale-candidate", sdpMid: "0", usernameFragment: "previous" };
|
||||
const current = { candidate: "current-candidate", sdpMid: "0", usernameFragment: "current" };
|
||||
for (const candidate of [stale, current]) {
|
||||
await peer.acceptSignal({ kind: "candidate", connectionId: peer.connectionId, candidate });
|
||||
}
|
||||
vi.spyOn(connection, "addIceCandidate").mockRejectedValueOnce(
|
||||
new DOMException("Unknown ICE username fragment", "OperationError"),
|
||||
);
|
||||
await peer.acceptSignal({ kind: "description", connectionId: peer.connectionId,
|
||||
description: { type: "answer", sdp: "current-answer" } });
|
||||
expect(connection.addedIceCandidates).toEqual([current]);
|
||||
expect(peer.getSnapshot().error).toBeNull();
|
||||
peer.dispose();
|
||||
});
|
||||
|
||||
it("enables the selected balanced profile after startup frames", async () => {
|
||||
const video = createTrack("video", "video");
|
||||
const peer = createPeer(createStream(video, createTrack("audio", "audio")));
|
||||
|
||||
@@ -58,7 +58,7 @@ describe("NAT prediction ICE adapter", () => {
|
||||
expect(isNativeNatSurveyCandidate(hostCandidate())).toBe(false);
|
||||
});
|
||||
|
||||
it("ignores only rejected predicted candidates", async () => {
|
||||
it("keeps unexpected ordinary-candidate failures visible", async () => {
|
||||
const error = new Error("candidate rejected");
|
||||
const connection = {
|
||||
addIceCandidate: vi.fn(async () => {
|
||||
@@ -82,6 +82,15 @@ describe("NAT prediction ICE adapter", () => {
|
||||
await expect(addRemoteIceCandidate(connection, null)).rejects.toBe(error);
|
||||
});
|
||||
|
||||
it("isolates candidate rejection without hiding a missing remote description", async () => {
|
||||
const connection = { addIceCandidate: vi.fn() } as unknown as RTCPeerConnection;
|
||||
vi.mocked(connection.addIceCandidate).mockRejectedValueOnce(new DOMException("Rejected", "OperationError"));
|
||||
await expect(addRemoteIceCandidate(connection, candidate(40_000))).resolves.toBeUndefined();
|
||||
const error = new DOMException("Remote description is missing", "InvalidStateError");
|
||||
vi.mocked(connection.addIceCandidate).mockRejectedValueOnce(error);
|
||||
await expect(addRemoteIceCandidate(connection, candidate(40_000))).rejects.toBe(error);
|
||||
});
|
||||
|
||||
it("leaves the configured servers unchanged when disabled", () => {
|
||||
const servers = [
|
||||
{ urls: ["stun:share.example.test:3478"] },
|
||||
|
||||
@@ -241,6 +241,24 @@ afterEach(() => {
|
||||
});
|
||||
|
||||
describe("ViewerPeer connection generations", () => {
|
||||
it("answers after a rejected queued candidate and still applies later candidates", async () => {
|
||||
const signals: SignalPayload[] = [];
|
||||
const snapshots: PeerSnapshot[] = [];
|
||||
const peer = createPeer(signals, snapshots);
|
||||
await peer.acceptSignal("host", candidate("queued", "stale-candidate"));
|
||||
const current = candidate("queued", "current-candidate");
|
||||
await peer.acceptSignal("host", current);
|
||||
const answering = peer.acceptSignal("host", offer("queued"));
|
||||
const connection = FakePeerConnection.instances[0]!;
|
||||
connection.addIceCandidate.mockRejectedValueOnce(new DOMException("Unknown ICE username fragment", "OperationError"));
|
||||
await answering;
|
||||
expect(connection.addIceCandidate).toHaveBeenCalledTimes(2);
|
||||
expect(signals).toHaveLength(1);
|
||||
expect(signals[0]).toMatchObject({ kind: "description", description: { type: "answer" } });
|
||||
expect(snapshots.at(-1)?.error).toBeNull();
|
||||
peer.dispose();
|
||||
});
|
||||
|
||||
it("keeps separately delivered audio and video on one remote stream", async () => {
|
||||
const streams: MediaStream[] = [];
|
||||
const peer = new ViewerPeer(
|
||||
|
||||
Reference in new issue
Block a user