diff --git a/docs/adr/0005-automatic-hybrid-media-routing.md b/docs/adr/0005-automatic-hybrid-media-routing.md index 75648b1..7402b61 100644 --- a/docs/adr/0005-automatic-hybrid-media-routing.md +++ b/docs/adr/0005-automatic-hybrid-media-routing.md @@ -246,9 +246,11 @@ as another application route candidate. ### Recovery, Pause, And Quality WebRTC and LiveKit first own ICE/consent and transient reconnect on the current -route. Only framework recovery exhaustion, hard failure, a non-paused decoded- -frame stall, confirmed departure, or capacity invalidation enters route -reconciliation. Manual reconnect remains on the exact current route. +route. Only framework recovery exhaustion, hard failure, confirmed departure, +or capacity invalidation enters route reconciliation. Decoded-frame silence +alone cannot locate a route failure: a live source can stop repainting, and +missing decoded frames do not distinguish source, decoder and transport faults. +Manual reconnect remains on the exact current route. The retained connection and a same-parent candidate keep separate signaling authority. A replacement offer from the retained parent supersedes optional preparation for that child at adoption, including an SFU bootstrap carrier's @@ -256,9 +258,9 @@ trial. This retires its reservation and fences late candidate signaling; required availability work keeps priority and candidate budgets do not reopen. Authoritative Host pause aborts the pending operation and reservations, preserves -the committed graph, suppresses decoded-stall authority, and leaves new Viewers -waiting. Resume starts reconciliation from the current graph. Page-hidden wall -time is rebaselined before it can contribute to a stall decision. +the committed graph, and leaves new Viewers waiting. Resume starts +reconciliation from the current graph. Missing or suspended observations do +not create route-failure authority. Native-edge convergence runs inside the same graph, reconcile loop, and room-serial child operation when its per-share gate is enabled. Availability diff --git a/docs/research/browser-local-encoding-pool.md b/docs/research/browser-local-encoding-pool.md index 70188e0..693741e 100644 --- a/docs/research/browser-local-encoding-pool.md +++ b/docs/research/browser-local-encoding-pool.md @@ -146,6 +146,27 @@ spikes and shared budget changes; a genuinely weaker child can still need its own producer. No delayed-budget policy, bitrate floor or manual recovery probe was added. Probe padding remains framework-owned traffic under native limits. +Matched short-pulse checks on Windows Chrome 152 used the same synthetic +1080p30 source, two children, played audio and `balanced` preference in both +paths: 45 healthy seconds, A constrained to 400 kbps for one second, then 40 +seconds of recovery. B was unconstrained. Each codec's ordinary/pool pair used +the same probe bundle; these are single-run observations, not averages. + +| Codec / path | A's lowest decoded size | A regained 1080p after pulse start | +| --- | --- | --- | +| VP8 / ordinary | 1280x720 | 20.2 s | +| VP8 / pool | 480x270 | 12.7 s | +| H264 / ordinary | 1280x720 | 23.3 s | +| H264 / pool | 1280x720 | 7.7 s | + +B retained 1080p throughout all four runs, with no reported connection failure. +The pooled traces created one separate producer for constrained A, then rejoined +the retained healthy producer; they did not show repeated producer churn. +The deeper VP8 dip remains a quality cost to investigate. These results do not +establish the cause of an unknown reporter's blur or stream loss, nor test Native +capture's separate keyframe-request path. Any further change must preserve +healthy-child isolation and framework-owned adaptation. + Budget attribution remains unchanged: targetBitrate is the encoder's allocated target, not raw link bandwidth ([upstream stats correction](https://webrtc.googlesource.com/src/+/fe25b0e928ea4e64aa134f5dc8012343320deec5%5E%21/)). Replacing it with availableOutgoingBitrate would bypass native allocation and diff --git a/docs/research/native-client-lifecycle.md b/docs/research/native-client-lifecycle.md index d36ddb0..f09fc6e 100644 --- a/docs/research/native-client-lifecycle.md +++ b/docs/research/native-client-lifecycle.md @@ -246,6 +246,32 @@ Aspect references: [active display paths](https://learn.microsoft.com/en-us/wind [active signal size](https://learn.microsoft.com/en-us/windows/win32/api/wingdi/ns-wingdi-displayconfig_video_signal_info), and [independent panel fitting/MPO](https://learn.microsoft.com/en-us/windows/win32/direct3ddxgi/for-best-performance--use-dxgi-flip-model). +## Quiet Sources And Viewer Recovery + +A later Windows 11/Chrome recheck used the installed CS2 in an offline map, +with H264 and VP8 window capture and H264 display capture. A minimized VP8 +source started with zero decoded frames for over ten seconds without ending +the share. H264 and VP8 profile changes remained pending beyond the old +five-second timeout, then delivered after restoration. This confirms that the +native quiet-source preparation repair remains in place. + +The 1920x1440 fullscreen window case exposed a separate Viewer rule: fifteen +seconds without a decoded frame sent an active `route-failed` despite connected +transport. This retired the Viewer route and its unproved quality candidate; +the Host source remained alive. That sample also recorded packet loss, so it +does not establish whether its original decode interruption came from source +silence, forwarding or decoding. The decoded-only failure rule is removed for +both P2P and SFU: silence cannot establish which owner failed. Framework +failure/recovery, first-frame admission and candidate proof remain unchanged. +The repeated fullscreen case retained the connection and resumed decoded +720p output after restoration. A controlled production-path check then held +Native output for 35 seconds without announcing Host pause: the visible Viewer +decoded no new frames, kept the same connected media, and decoded again when +output resumed. This is bounded device evidence, not proof of +all game/driver recovery or the separate immediate share-start failure report. +Some focus changes did not remain foreground throughout a sampling window; +only recorded window states and receiver progress establish those transitions. + ## Windows Capture Borders The accepted scope keeps WGC for Windows capture and previews. Piik does not diff --git a/docs/standards/media-quality.md b/docs/standards/media-quality.md index 863dc28..657bf53 100644 --- a/docs/standards/media-quality.md +++ b/docs/standards/media-quality.md @@ -204,6 +204,9 @@ needed output and its lower fallbacks and stops unused upper outputs. Each child receives only its selected output. The forwarding SFU adds no server transcoder. Downstream attachment or sending failure and retirement belong to that edge, never its shared video/audio source or healthy siblings. +Native keyframe requests retain their output scope through coalescing; a relay +requests its single upstream stream once per batch while keeping local output +recovery separate. Native capture uses the same screen-share output construction within Host ceilings. Its source, source clock and higher outputs remain independent of diff --git a/docs/standards/presentation-lifecycle.md b/docs/standards/presentation-lifecycle.md index 52c1e86..5c35540 100644 --- a/docs/standards/presentation-lifecycle.md +++ b/docs/standards/presentation-lifecycle.md @@ -59,6 +59,11 @@ borrowed, never reopened or stopped by preview. Optional native ingress uses an available connection without another scan. If it is not ready at Browser startup, that share keeps Browser senders. +Native Host startup acquires its required Browser media connection before +starting capture. Cancellation owns that connection immediately, while native +share ownership and required audio tracks come from the accepted App response. +Browser initialization failure remains distinct from capture permission failure. + A reachable Piik App with an incompatible control protocol is distinct from an absent App. Discovery continues looking for a compatible process before reporting the mismatch; the source selector retains Browser operation and offers update/ @@ -207,9 +212,12 @@ black screen, ICE-connected state, or unproved `playing` event. - An actionable overlay may appear while fullscreen remains active; application state never forces the user out of fullscreen. -Visibility, page freeze, and pagehide suppress application decoded-stall -authority and invalidate quality observations, not already-proved playback. -Returning to the page rebaselines time and re-arms frame observation; an existing +Decoded-frame silence is not a route-failure signal. A live source can stop +repainting; media-framework failures, not an application silence timer, own +recovery. First-frame admission and candidate-relative proof still require +decoded progress. Visibility, page freeze, and pagehide invalidate quality +observations, not already-proved playback. +Returning to the page re-arms frame observation; an existing recovery state still requires a fresh current frame to clear. SFU media is retained independently of transient room-signaling loss, but Browser or OS suspension and page reclamation remain outside Web guarantees. diff --git a/docs/standards/visual-language.md b/docs/standards/visual-language.md index fda7bb3..39d5f74 100644 --- a/docs/standards/visual-language.md +++ b/docs/standards/visual-language.md @@ -50,6 +50,11 @@ Hover, keyboard focus and touch receive equivalent guidance. Keep information order and interaction ownership consistent across languages, themes and viewport sizes. Reflow or bounded scrolling must not hide a primary action, truncate an essential value, overlap controls or change meaning. +Keep a compact value and its direct actions together when rows wrap, including +the room code and its replacement confirmation. Let independent groups reflow; +do not stretch short values to fill a phone row. Check actual control bounds +around breakpoints, in both orientations and all presentation modes; page-level +overflow clipping is not evidence that the content fits. Keep the application language control's segmented Chinese, English and visual shortcuts. When additional catalogs exist, insert one language menu @@ -80,6 +85,8 @@ the whole playback bar. A wrapped option row is also one area to avoid. Status, headers and bottom-of-picker controls prefer below; other controls prefer above. Flip to fit the viewport and keep the caret aimed at the trigger. Placement and hover timing belong to the shared component. +Constrain oversized hints to the visible viewport and let their content scroll +without dismissing the panel or clipping its caret. Add a tooltip when it explains an action, a limitation or a status that needs context. Plain names, decorative participants and self-explanatory text do not diff --git a/docs/todo.md b/docs/todo.md index 705deaa..e63135a 100644 --- a/docs/todo.md +++ b/docs/todo.md @@ -1,12 +1,21 @@ # Current TODO Ledger -Last reviewed: 2026-09-25 +Last reviewed: 2026-09-27 Only **Now** is executable. Product modules own behavior; Git/PRs own completed history. A parked idea is not implementation authority. ## Now +- [ ] **Investigate Browser short-pulse quality cost.** Matched ordinary/pool + checks with two 1080p30 children, audio and a one-second bandwidth constraint + reproduce a deeper VP8 dip in the pool (270p versus 720p), with faster recovery. + H264 reaches 720p on both paths; unaffected children stay at 1080p. The + [pool evidence](./research/browser-local-encoding-pool.md#balanced-startup-and-recovery) + distinguishes one cold producer handoff from repeated churn. Assess a smaller + adaptation cost without a custom bitrate floor, delay policy or weakening + healthy-child isolation. These bounded Browser results do not identify the + unknown reporters' capture paths or establish their causes. - [ ] **Passive App attachment: design hold.** Site mode authorizes one selected origin and supplies native media without starting a local room server. A passive replacement needs an accepted site-consent/discovery flow; it must not @@ -90,7 +99,11 @@ history. A parked idea is not implementation authority. retired event delivery and SFU replacement have locally reproduced defects and regression checks, but paired diagnostics and device/network details are still needed to establish this reporter's cause. Include an upstream relay's - report when present. The existing five-second Viewer membership grace during + report when present. Terminal Browser sender failures now notify their existing + preparation owner instead of silently disposing; expected retirement stays + silent. Native per-output keyframe requests no longer wake unrelated outputs. + These are confirmed local defects, not matched causes of the field reports. + The existing five-second Viewer membership grace during signaling loss is unchanged. This is distinct from the first-frame report below. - [ ] **Windows 11 capture border remains visible.** Identify the App/Browser capture path, Windows build and capture-border permission result. Local checks @@ -108,31 +121,32 @@ history. A parked idea is not implementation authority. - [ ] **App discovery and share-start field acceptance.** Retest the missing-window report, an unreachable App despite its process running, and generic share-start failure. Distinguish site authorization, browser permission, control capacity, - enumeration and capture startup. A complete v1.6.3 Windows report confirms - Auto/Window/1080p30 balanced startup: hardware MFT event timeout, then software - probing, then cancellation because capture readiness did not arrive. The stall - within probing, teardown or WGC initialization remains unknown; one initial - VP8 statistics sample cannot establish its complete output history. Auto now - skips the redundant comparison when only VP8 remains. Injected unavailable - hardware and exhausted-probe-budget cases both capture and decode locally; - the affected machine still needs same-source Auto versus explicit VP8 - acceptance. Do not extend deadlines without identifying the blocked stage. - Existing H264-retention/ICE-candidate repairs do not establish this cause; other - discovery/manual-H264 reports still need paired App/Browser diagnostics. + enumeration and capture startup. The v1.6.5 report reaches Auto-to-VP8 startup + after a hardware MFT timeout, then receives Browser `stop-share` before any + local-edge request. App handler timestamps are about 1.5 ms apart; this is not + a Browser-measured ACK interval or a no-frame deadline. Deliberately exhausting + real Chrome connection objects reproduces that sequence with physical capture; + bounded normal restarts showed no retained connections. Startup now acquires + the Browser connection first: the same exhaustion rejects before capture, and + releasing fixture-held objects permits sharing again. This establishes neither + a Piik resource leak nor the reporter's cause. The actual Browser exception, + loaded asset identity and reporter VP8 output remain unknown. Zero dimensions/ + FPS in `starting` are valid. Diagnostics preserve preview initialization/ + negotiation causes and distinguish MFT input/output waits; obtain matched + evidence before changing recovery or deadlines. Other discovery/manual-H264 + reports still need paired evidence. - [ ] **Share ends after entering a game.** Screen sharing reportedly works until entering a game freezes the picture, followed seconds later by share termination. Version, capture path, codec and matched diagnostics are unknown. Locate the first capture/output, preview-bridge, control or authority failure; distinguish ordinary source silence/resize from target replacement, exclusive fullscreen, display-mode change and device loss. A reproduced downstream - retirement race no longer stops the shared source. Controlled-window checks - do not reproduce the reporting game/device environment. -- [ ] **Brief blur followed by immediate recovery.** Compare the same source, - codec, peers and network with ordinary WebRTC before attributing the report - to pooling. The [pool checks](./research/browser-local-encoding-pool.md#balanced-startup-and-recovery) - cover rate-owner retention, publication startup, native recovery probing and - ordinary fallback. They do not establish this reporter's capture path or result; - correlate actual receiver dimensions, QP, resource adaptation and packet loss. + retirement race no longer stops the shared source. Actual CS2 checks found + and removed a separate decoded-silence rule that retired a connected Viewer + route while the Host stayed live; [capture research](./research/native-client-lifecycle.md#quiet-sources-and-viewer-recovery) + distinguishes that repair and successful restoration from the unknown cause + of the reported Host termination. Do not conflate source silence, receiver + decode interruption and capture-process failure. - [ ] **Windows 32-bit candidate acceptance.** Verify the isolated `spike/windows-x86-capture` candidate's launch, capture/audio, memory pressure, source replacement and update links on a 32-bit Windows device. WOW64 diff --git a/internal/app/mediaedge/edge_test.go b/internal/app/mediaedge/edge_test.go index 8dba41f..6b5a495 100644 --- a/internal/app/mediaedge/edge_test.go +++ b/internal/app/mediaedge/edge_test.go @@ -37,7 +37,7 @@ func TestOutputPlanRetiresOnlyFailedLayerConsumers(t *testing.T) { } t.Cleanup(func() { _ = engine.Close() }) wake := make(chan struct{}, 8) - source, err := engine.NewSource("vp8", 2, 2, func() { wake <- struct{}{} }) + source, err := engine.NewSource("vp8", 2, 2, func([]int) { wake <- struct{}{} }) if err != nil { t.Fatal(err) } @@ -263,7 +263,7 @@ func testEncodedSourceFanout(t *testing.T, codec string) { } t.Cleanup(func() { _ = engine.Close() }) keyFrames := make(chan struct{}, 1) - source, err := engine.NewSource(codec, 2, 1, func() { + source, err := engine.NewSource(codec, 2, 1, func([]int) { select { case keyFrames <- struct{}{}: default: diff --git a/internal/app/mediaedge/engine.go b/internal/app/mediaedge/engine.go index eacfd11..196418c 100644 --- a/internal/app/mediaedge/engine.go +++ b/internal/app/mediaedge/engine.go @@ -267,7 +267,9 @@ func (engine *Engine) ListenAddress() string { return engine.listenAddress } -func (engine *Engine) NewSource(codec string, capacity, layers int, requestKeyFrame func()) (*Source, error) { +// NewSource coalesces recovery requests by physical output slot; [-1] requests +// all outputs. The callback runs outside the source lock and forwarding path. +func (engine *Engine) NewSource(codec string, capacity, layers int, requestKeyFrame func([]int)) (*Source, error) { _, supported := videoCodecs[codec] if !supported { return nil, errors.New("native video codec is unsupported") diff --git a/internal/app/mediaedge/groups.go b/internal/app/mediaedge/groups.go index c06620d..15f3c81 100644 --- a/internal/app/mediaedge/groups.go +++ b/internal/app/mediaedge/groups.go @@ -237,11 +237,10 @@ func (source *Source) newGroup(slot int) (*outputGroup, error) { switch packet.(type) { case *rtcp.PictureLossIndication, *rtcp.FullIntraRequest: if layer == 0 { - source.groupRecovery.Or(uint32(1) << slot) + source.requestLayerKeyFrame(slot) } else { - source.groupRecovery.Or(2) + source.requestLayerKeyFrame(1) } - source.RequestRecoveryFrame() } } }, diff --git a/internal/app/mediaedge/receiver.go b/internal/app/mediaedge/receiver.go index 74c28dd..5b3f38d 100644 --- a/internal/app/mediaedge/receiver.go +++ b/internal/app/mediaedge/receiver.go @@ -82,7 +82,10 @@ func (engine *Engine) NewReceiver(options ReceiverOptions) (*Receiver, webrtc.Se if _, supported := relayBackend(videoCodec, options.Relay); supported { layers = 2 } - receiver.source, err = engine.NewSource(videoCodec, options.EdgeCapacity, layers, receiver.RequestKeyFrame) + receiver.source, err = engine.NewSource(videoCodec, options.EdgeCapacity, layers, func([]int) { + // Local output slots all derive from this one upstream encoded stream. + receiver.RequestKeyFrame() + }) if err != nil { _ = connection.Close() return nil, webrtc.SessionDescription{}, err diff --git a/internal/app/mediaedge/source.go b/internal/app/mediaedge/source.go index a6db3c8..ff0dac7 100644 --- a/internal/app/mediaedge/source.go +++ b/internal/app/mediaedge/source.go @@ -20,6 +20,7 @@ const videoClockRate = 90_000 const videoPacketMTU = 1200 const h264PayloadType = 102 const vp8PayloadType = 96 +const allRecoveryLayers = ^uint32(0) var ErrSourceCapacity = errors.New("native media source capacity is exhausted") @@ -38,12 +39,13 @@ type Source struct { formats []atomic.Uint64 outputBitrates []uint32 capacity int - requestKeyFrame func() + requestKeyFrame func([]int) relayProfile *nativecapture.VideoProfile relay *relayDerivation groups map[int]*outputGroup memberships map[groupConsumer]*outputGroup groupRecovery atomic.Uint32 + pendingRecovery atomic.Uint32 recoveryRequests chan struct{} recoveryStopped chan struct{} inputPTS time.Duration @@ -607,11 +609,22 @@ func (source *Source) detach(edge *Edge) { func (source *Source) requestLayerKeyFrame(layer int) { source.groupRecovery.Or(uint32(1) << layer) - source.RequestRecoveryFrame() + source.requestRecovery(layer) } func (source *Source) RequestRecoveryFrame() { + source.requestRecovery(-1) +} + +func (source *Source) requestRecovery(layer int) { if source.requestKeyFrame != nil { + mask := allRecoveryLayers + if layer >= 0 { + mask = uint32(1) << layer + } + // Coalesce notifications without dropping another output's request. + // groupRecovery separately belongs to the next input's output plan. + source.pendingRecovery.Or(mask) select { case source.recoveryRequests <- struct{}{}: default: @@ -637,7 +650,20 @@ func (source *Source) runRecovery() { closed := source.closed source.mu.Unlock() if !closed { - source.requestKeyFrame() + mask := source.pendingRecovery.Swap(0) + if mask == 0 { + continue + } + layers := []int{-1} + if mask != allRecoveryLayers { + layers = nil + for layer := 0; mask != 0; mask, layer = mask>>1, layer+1 { + if mask&1 != 0 { + layers = append(layers, layer) + } + } + } + source.requestKeyFrame(layers) } } } diff --git a/internal/app/mediaedge/source_test.go b/internal/app/mediaedge/source_test.go index d077bbe..ba2faf6 100644 --- a/internal/app/mediaedge/source_test.go +++ b/internal/app/mediaedge/source_test.go @@ -4,6 +4,7 @@ import ( "errors" "fmt" "io" + "slices" "testing" "time" @@ -15,6 +16,92 @@ import ( "google.golang.org/protobuf/proto" ) +func TestSourceRecoveryCoalescesTargetsWithoutWakingUnrelatedOutputs(t *testing.T) { + for _, all := range []bool{false, true} { + t.Run(fmt.Sprint(all), func(t *testing.T) { + requested := make(chan []int, 8) + // Queue several requests before starting the existing notification worker. + source := &Source{engine: &Engine{}, requestKeyFrame: func(layers []int) { requested <- layers }, + recoveryRequests: make(chan struct{}, 1), recoveryStopped: make(chan struct{})} + source.requestLayerKeyFrame(0) + source.requestLayerKeyFrame(2) + source.requestLayerKeyFrame(0) + if all { + source.RequestRecoveryFrame() + } + done := make(chan struct{}) + go func() { defer close(done); source.runRecovery() }() + defer func() { close(source.recoveryStopped); <-done }() + want := []int{0, 2} + if all { + want = []int{-1} + } + select { + case layers := <-requested: + if !slices.Equal(layers, want) { + t.Fatalf("requested outputs %v instead of %v", layers, want) + } + case <-time.After(time.Second): + t.Fatal("lost output recovery request") + } + if source.groupRecovery.Load() != 5 { + t.Fatal("wake callback consumed the frame-boundary recovery plan") + } + select { + case layers := <-requested: + t.Fatalf("unexpected extra callback: %v", layers) + default: + } + }) + } +} + +func TestSourcePLIUsesPhysicalOutputIdentity(t *testing.T) { + requested := make(chan []int, 16) + source, err := (&Engine{}).NewSource("vp8", 2, 2, func(layers []int) { requested <- layers }) + if err != nil { + t.Fatal(err) + } + defer source.Close() + if err = source.ConfigureOutputs([]uint32{300_000, 1_200_000, 300_000}); err != nil { + t.Fatal(err) + } + if _, err = source.planGroups(nil); err != nil { + t.Fatal(err) + } + group, err := source.newGroup(2) + if err != nil { + t.Fatal(err) + } + for _, media := range []*forwarding.EncodedSource{source.media, group.media} { + if err = media.BeginFrame(time.Second, time.Now()); err != nil { + t.Fatal(err) + } + for layer := range 2 { + if err = media.WriteFrame(layer, encoded.Frame{Data: sfu.VP8KeyFrame8x8, PTS: time.Second, + Duration: time.Second / 30, Recovery: true}); err != nil { + t.Fatal(err) + } + } + } + for _, request := range []struct { + media *forwarding.EncodedSource + layer, physical int + }{ + {source.media, 0, 0}, {source.media, 1, 1}, {group.media, 0, 2}, {group.media, 1, 1}, + } { + request.media.SendPLI(int32(request.layer), true) + select { + case layers := <-requested: + if !slices.Equal(layers, []int{request.physical}) { + t.Fatalf("logical %d requested physical %v, want %d", request.layer, layers, request.physical) + } + case <-time.After(time.Second): + t.Fatal("PLI did not wake its output") + } + } +} + func TestSourceFrameSurvivesRetiringEdge(t *testing.T) { for _, layers := range []int{1, 2} { t.Run(fmt.Sprint(layers), func(t *testing.T) { diff --git a/internal/app/nativehost/session.go b/internal/app/nativehost/session.go index 6c792cf..2aed1f7 100644 --- a/internal/app/nativehost/session.go +++ b/internal/app/nativehost/session.go @@ -748,9 +748,11 @@ func (session *Session) runVideo() error { session.mu.Unlock() return fail(errors.New("native capture codec was not applied")) } - source, sourceErr := session.engine.NewSource(status.Codec, session.edgeCapacity, min(2, len(status.Outputs)), func() { + source, sourceErr := session.engine.NewSource(status.Codec, session.edgeCapacity, min(2, len(status.Outputs)), func(layers []int) { if stream := session.currentStream(); stream != nil { - _ = stream.RequestKeyFrame(-1) + for _, layer := range layers { + _ = stream.RequestKeyFrame(layer) + } } }) if sourceErr != nil { diff --git a/native/capture/windows/h264_encoder.cpp b/native/capture/windows/h264_encoder.cpp index c58dd9a..0b8febb 100644 --- a/native/capture/windows/h264_encoder.cpp +++ b/native/capture/windows/h264_encoder.cpp @@ -657,7 +657,7 @@ EncodedAccessUnit LiveEncoder::Encode( const auto deadline = std::min(probe_deadline, EncoderClock::now() + std::chrono::seconds(2)); while (std::chrono::steady_clock::now() < deadline) { - MediaEventType type = NextEvent(deadline); + MediaEventType type = NextEvent(deadline, "mft-output-timeout"); if (type == METransformNeedInput) { ++input_requests_; continue; @@ -699,7 +699,8 @@ EncodedAccessUnit LiveEncoder::Encode( Fail("mft-output-timeout", "hardware MFT did not produce live output in time"); } -MediaEventType LiveEncoder::NextEvent(std::chrono::steady_clock::time_point deadline) { +MediaEventType LiveEncoder::NextEvent( + EncoderClock::time_point deadline, const char* timeout_stage) { while (std::chrono::steady_clock::now() < deadline) { ComPtr event; HRESULT result = selected_.events->GetEvent(MF_EVENT_FLAG_NO_WAIT, @@ -717,14 +718,14 @@ MediaEventType LiveEncoder::NextEvent(std::chrono::steady_clock::time_point dead if (type == MEError) Fail("mft-error-event", "hardware MFT emitted MEError"); return type; } - Fail("mft-event-timeout", "hardware MFT event wait timed out"); + Fail(timeout_stage, "hardware MFT event wait timed out"); } void LiveEncoder::WaitForInput(EncoderClock::time_point probe_deadline) { const auto deadline = std::min(probe_deadline, EncoderClock::now() + std::chrono::seconds(2)); while (input_requests_ == 0) { - MediaEventType type = NextEvent(deadline); + MediaEventType type = NextEvent(deadline, "mft-input-timeout"); if (type == METransformNeedInput) { ++input_requests_; } else if (type == METransformHaveOutput) { diff --git a/native/capture/windows/h264_encoder.h b/native/capture/windows/h264_encoder.h index 0b2d8ae..aabaa94 100644 --- a/native/capture/windows/h264_encoder.h +++ b/native/capture/windows/h264_encoder.h @@ -225,7 +225,7 @@ class LiveEncoder final : public VideoEncoder { EncoderClock::time_point deadline = EncoderClock::time_point::max()) override; private: - MediaEventType NextEvent(EncoderClock::time_point deadline); + MediaEventType NextEvent(EncoderClock::time_point deadline, const char* timeout_stage); void WaitForInput(EncoderClock::time_point deadline); SelectedTransform selected_; diff --git a/site/film/film.css b/site/film/film.css index 6e45846..3c80421 100644 --- a/site/film/film.css +++ b/site/film/film.css @@ -22,7 +22,8 @@ button,a,input { outline-offset:5px; } .masthead nav a,.masthead button,#language { min-height:44px; display:inline-flex; align-items:center; text-decoration:none; } .masthead button { font-size:12px; } .masthead nav :is(a,button):hover { color:var(--accent); } -main { width:calc(100% - 64px); max-width:min(1664px,calc((100svh - 248px) * 16 / 9)); margin:auto; } +/* A short landscape viewport may scroll; it must not shrink the player controls. */ +main { width:calc(100% - 64px); max-width:clamp(640px,calc((100svh - 248px) * 16 / 9),1664px); margin:auto; } .cinema { position:relative; background:#f7f4e9; color:#22303e; border-radius:24px; box-shadow:0 6px 0 #22303e12; overflow:hidden; isolation:isolate; } .picture { position:relative; aspect-ratio:16/9; } #film-art { display:none; width:100%; height:100%; } diff --git a/site/styles.css b/site/styles.css index f7d7925..3f0f6f0 100644 --- a/site/styles.css +++ b/site/styles.css @@ -86,6 +86,9 @@ h1 { font-size: clamp(49px, 5.7vw, 76px); line-height: 1.06; margin-block: 22px .film-preview > summary::after { content:none; } .film-collapse { margin-left:auto; font-size:12px; color:var(--muted); } .film-preview iframe { display:block; width:100%; height:calc(56.25cqw + 144px); border:0; } +@container (max-width:740px) { + .film-preview iframe { height:calc(56.25cqw + 196px); } +} .download-help { margin-top:22px; text-align:center; } .download-links { display:flex; justify-content:center; align-items:center; flex-wrap:wrap; gap:12px 24px; margin-bottom:14px; } .download-help p { max-width:760px; margin:6px auto 0; color:var(--muted); font-size:13px; } @@ -214,7 +217,6 @@ html[lang="zh-CN"] .eyebrow { letter-spacing: .13em; } .footer-identity p { display: none; } } @media (max-width: 740px) { - .film-preview iframe { height:calc(56.25cqw + 196px); } .wrap { width: calc(100% - 40px); } .site-header { min-height: 83px; gap: 0 14px; flex-wrap:wrap; padding-top:12px; } .site-header nav { order:3; flex-basis:100%; justify-content:space-between; gap:0 12px; } diff --git a/src/client/components/living/RoomChip.tsx b/src/client/components/living/RoomChip.tsx index f5ac577..d8a0769 100644 --- a/src/client/components/living/RoomChip.tsx +++ b/src/client/components/living/RoomChip.tsx @@ -130,7 +130,7 @@ export function RoomChip({ } return ( - <> + {onReplace ? ( confirming ? ( @@ -161,7 +161,7 @@ export function RoomChip({ /> ) ) : null} - + ); } diff --git a/src/client/components/living/Tooltip.tsx b/src/client/components/living/Tooltip.tsx index 912c10a..b673c14 100644 --- a/src/client/components/living/Tooltip.tsx +++ b/src/client/components/living/Tooltip.tsx @@ -162,8 +162,14 @@ export function Tooltip({ const pickAlign = () => { const wrap = wrapRef.current; if (!wrap) return; - const vw = window.innerWidth; + const viewport = window.visualViewport; + const vw = viewport?.width ?? window.innerWidth; if (!vw) return; // no layout (SSR/test): keep the prop alignment + const vh = viewport?.height ?? window.innerHeight; + const x = viewport?.offsetLeft ?? 0, y = viewport?.offsetTop ?? 0; + const right = x + vw, bottom = y + vh; + tipRef.current?.style.setProperty("--tooltip-max-width", `${Math.max(0, vw - EDGE_MARGIN * 2)}px`); + tipRef.current?.style.setProperty("--tooltip-max-height", `${Math.max(0, vh - EDGE_MARGIN * 2)}px`); const rect = wrap.getBoundingClientRect(); // A wrapped control row remains one working area, including on narrow screens. const controlBar = wrap.closest(".lr-playback, .lr-source-picker-options, .lr-host-share-controls"); @@ -177,11 +183,10 @@ export function Tooltip({ const width = Math.max(0, Math.min(measured || 320, vw - EDGE_MARGIN * 2)); // Same idea vertically: a control scrolled near the top has no room above, // and the panel would be cut off by the viewport edge. - const vh = window.innerHeight; const panelHeight = (panelMounted ? tipRef.current?.offsetHeight : 0) || FALLBACK_PANEL_HEIGHT; - const fitsAbove = avoid.top - panelHeight - EDGE_MARGIN >= EDGE_MARGIN; - const fitsBelow = !vh || below + panelHeight + EDGE_MARGIN <= vh - EDGE_MARGIN; + const fitsAbove = avoid.top - panelHeight - EDGE_MARGIN >= y + EDGE_MARGIN; + const fitsBelow = !vh || below + panelHeight + EDGE_MARGIN <= bottom - EDGE_MARGIN; const preferredPlace = wrap.closest(".lr-tv-chin, .lr-host-share-controls") ? "below" : place; let livePlace: Placement = preferredPlace === "above" @@ -192,8 +197,8 @@ export function Tooltip({ ? "below" : "above"; const centerY = rect.top + rect.height / 2; - const sideTop = Math.max(EDGE_MARGIN, Math.min(centerY - panelHeight / 2, vh - panelHeight - EDGE_MARGIN)); - const fitsSideVertically = sideTop >= EDGE_MARGIN && sideTop + panelHeight <= (vh || Infinity) - EDGE_MARGIN; + const sideTop = Math.max(y + EDGE_MARGIN, Math.min(centerY - panelHeight / 2, bottom - panelHeight - EDGE_MARGIN)); + const fitsSideVertically = sideTop >= y + EDGE_MARGIN && sideTop + panelHeight <= (bottom || Infinity) - EDGE_MARGIN; // Use empty side space only when the whole panel fits close to its control. // Otherwise clear the entire playback bar, including its narrow second row. if (playback) { @@ -202,8 +207,8 @@ export function Tooltip({ const sideAvoid = player ?? avoid; const nearLeft = rect.left - sideAvoid.left <= rect.width; const nearRight = sideAvoid.right - rect.right <= rect.width; - const fitsLeft = nearLeft && sideAvoid.left - width - EDGE_MARGIN >= EDGE_MARGIN; - const fitsRight = nearRight && sideAvoid.right + width + EDGE_MARGIN <= vw - EDGE_MARGIN; + const fitsLeft = nearLeft && sideAvoid.left - width - EDGE_MARGIN >= x + EDGE_MARGIN; + const fitsRight = nearRight && sideAvoid.right + width + EDGE_MARGIN <= right - EDGE_MARGIN; // Prefer space below the television (including its status strip) to // covering the picture. Fullscreen falls back above the entire bar. livePlace = fitsSideVertically && (fitsLeft || fitsRight) ? fitsLeft ? "left" : "right" @@ -216,16 +221,17 @@ export function Tooltip({ end: { left: rect.right - width, right: rect.right }, }; const clipped = (box: { left: number; right: number }): number => - Math.max(0, EDGE_MARGIN - box.left) + - Math.max(0, box.right - (vw - EDGE_MARGIN)); + Math.max(0, x + EDGE_MARGIN - box.left) + + Math.max(0, box.right - (right - EDGE_MARGIN)); const order: Align[] = [ align, ...(["center", "start", "end"] as Align[]).filter((a) => a !== align), ]; const fits = order.find((a) => clipped(boxes[a]) === 0); const selected = fits ?? order.reduce((a, b) => (clipped(boxes[a]) <= clipped(boxes[b]) ? a : b)); - let left = Math.max(EDGE_MARGIN, Math.min(boxes[selected].left, vw - width - EDGE_MARGIN)); + let left = Math.max(x + EDGE_MARGIN, Math.min(boxes[selected].left, right - width - EDGE_MARGIN)); let top = livePlace === "below" ? below + EDGE_MARGIN : avoid.top - panelHeight - EDGE_MARGIN; + top = Math.max(y + EDGE_MARGIN, Math.min(top, bottom - panelHeight - EDGE_MARGIN)); let caret = Math.max(14, Math.min(center - left, width - 14)); if (livePlace === "left") { left = (player ?? avoid).left - width - EDGE_MARGIN; @@ -270,10 +276,15 @@ export function Tooltip({ frame = window.requestAnimationFrame(() => pickAlignRef.current()); }; schedulePick(); + const viewport = window.visualViewport; + viewport?.addEventListener("resize", schedulePick); + viewport?.addEventListener("scroll", schedulePick); window.addEventListener("resize", schedulePick); window.addEventListener("scroll", schedulePick, true); return () => { window.cancelAnimationFrame(frame); + viewport?.removeEventListener("resize", schedulePick); + viewport?.removeEventListener("scroll", schedulePick); window.removeEventListener("resize", schedulePick); window.removeEventListener("scroll", schedulePick, true); }; @@ -413,6 +424,13 @@ export function Tooltip({ // completed long-press. A later touch starts a new, actionable gesture. longPressed.current = false; cancelPress(); + if (tipRef.current?.contains(event.target as Node)) { + // Reading/scrolling the panel is not another trigger long-press. + if (hideTimer.current !== null) window.clearTimeout(hideTimer.current); + hideTimer.current = null; + pressPoint.current = null; + return; + } touchGesture.current = event.pointerType === "touch"; if (!enabled || event.pointerType !== "touch") return; pressPoint.current = { x: event.clientX, y: event.clientY }; @@ -436,7 +454,7 @@ export function Tooltip({ }, TOUCH_HIDE_MS); } }} - onPointerCancel={() => { + onPointerCancel={(event) => { cancelPress(); pressPoint.current = null; longPressed.current = false; @@ -444,7 +462,7 @@ export function Tooltip({ window.clearTimeout(hideTimer.current); hideTimer.current = null; } - setPressOpen(false); + if (!tipRef.current?.contains(event.target as Node)) setPressOpen(false); }} onPointerMove={(event) => { // Only a real drag (scroll intent) cancels the hold — Chrome emits @@ -514,12 +532,12 @@ export function Tooltip({ data-tone={resolvedTone} style={{ ...comicStyle(resolvedTone, resolvedMotion), "--comic-repeat": "infinite", left: position.left, top: position.top, "--tooltip-caret": `${position.caret}px` } as CSSProperties} className={`lr-comic-tip${kind ? " has-comic" : ""}${caption !== undefined ? " is-text" : ""}${placeClass}`} role="tooltip" aria-hidden={!interactionOpen}> - {panelMounted ? <> + {panelMounted ? {kind ? isHintKind(kind) ? : : null} {caption !== undefined ? {caption} : null} - : null} + : null} ); diff --git a/src/client/locales/en.ts b/src/client/locales/en.ts index 6931b6d..9564656 100644 --- a/src/client/locales/en.ts +++ b/src/client/locales/en.ts @@ -141,6 +141,7 @@ export const en: Record = { "host.sourcePicker.failed": "Could not read the App's screen and window list. Refresh to try again; if it keeps failing, collect a Debug report.", "native.incompatible": "Piik App and this page are incompatible. Update Piik App and reload this page, or use Browser.", "native.fail.edge": "Piik App media connection failed", + "native.fail.browserMedia": "The browser could not create a media connection. Refresh the page or try another browser.", "native.fail.signaling": "Piik App signaling failed", "native.fail.receiver": "Piik App media receive failed", "host.sourcePicker.window": "Share window: {title}", diff --git a/src/client/locales/zh.ts b/src/client/locales/zh.ts index 1eac569..e90dc65 100644 --- a/src/client/locales/zh.ts +++ b/src/client/locales/zh.ts @@ -141,6 +141,7 @@ export const zh = { "host.sourcePicker.failed": "未能读取 App 的窗口和屏幕列表。请刷新重试；若持续失败，请导出诊断报告。", "native.incompatible": "Piik App 与当前页面不兼容。请更新 Piik App 并刷新页面，或使用浏览器。", "native.fail.edge": "Piik App 媒体连接失败", + "native.fail.browserMedia": "浏览器无法建立媒体连接。请刷新页面，或换一个浏览器重试。", "native.fail.signaling": "Piik App 信令失败", "native.fail.receiver": "Piik App 媒体接收失败", "host.sourcePicker.window": "分享窗口：{title}", diff --git a/src/client/media/decoded-frame-stall.ts b/src/client/media/decoded-frame-stall.ts deleted file mode 100644 index cb31378..0000000 --- a/src/client/media/decoded-frame-stall.ts +++ /dev/null @@ -1,62 +0,0 @@ -export const ROUTE_DECODED_FRAME_STALL_MS = 15_000; - -export class DecodedFrameStallDetector { - private identity: string | null = null; - private lastDecodedAt = 0; - private reported = false; - private paused = false; - - setPaused(paused: boolean, nowMs = Date.now()): void { - if (this.paused === paused) return; - this.paused = paused; - this.rebaseline(nowMs); - } - - rebaseline(nowMs = Date.now()): void { - this.lastDecodedAt = nowMs; - this.reported = false; - } - - allowReportRetry(): void { - this.reported = false; - } - - observe( - identity: string, - framesDecodedDelta: number | null, - nowMs = Date.now(), - ): boolean { - if (identity !== this.identity) { - this.identity = identity; - this.lastDecodedAt = nowMs; - this.reported = false; - return false; - } - if (this.paused) { - this.lastDecodedAt = nowMs; - this.reported = false; - return false; - } - if (framesDecodedDelta === null) { - this.lastDecodedAt = nowMs; - this.reported = false; - return false; - } - if (framesDecodedDelta > 0) { - this.lastDecodedAt = nowMs; - this.reported = false; - return false; - } - if (this.reported || nowMs - this.lastDecodedAt < ROUTE_DECODED_FRAME_STALL_MS) { - return false; - } - this.reported = true; - return true; - } - - reset(): void { - this.identity = null; - this.lastDecodedAt = 0; - this.reported = false; - } -} diff --git a/src/client/media/viewer-sfu-route.ts b/src/client/media/viewer-sfu-route.ts index 8341ab7..42715c0 100644 --- a/src/client/media/viewer-sfu-route.ts +++ b/src/client/media/viewer-sfu-route.ts @@ -34,7 +34,6 @@ interface ViewerSubscriberTransport { } interface ViewerSubscriberSlot { - mediaIdentity: string; connectionId: string; revision: number; phase: MediaRoutePhase; @@ -82,12 +81,6 @@ interface ViewerSfuRouteEvents { onSfuUpdate?: (metrics: ConnectionMetrics | null, revision: number) => void; currentPeerMetrics?: () => ConnectionMetrics | null; qualityProbeEligible?: () => boolean; - onSfuDecodedFrameSample?: ( - framesDecodedDelta: number | null, - revision: number, - mediaIdentity: string, - connectionId: string, - ) => void; onSfuState?: ( state: "connected" | "reconnecting", revision: number, @@ -98,7 +91,6 @@ interface ViewerSfuRouteEvents { onStream: (stream: MediaStream | null) => void; onVideoAvailability: (available: boolean) => void; onStats: (metrics: ConnectionMetrics) => void; - onDecodedFrameSample: (framesDecodedDelta: number | null) => void; onFirstDecodedFrame: () => boolean; onState: (state: "connected" | "reconnecting") => void; onDisconnected: () => void; @@ -115,7 +107,6 @@ export class ViewerSfuRoute { private transitionTail: Promise = Promise.resolve(); private resyncGeneration = 0; private resyncing = false; - private subscriberGeneration = 0; private paused = false; private closed = false; @@ -469,16 +460,6 @@ export class ViewerSfuRoute { } } }, - onDecodedFrameSample: (framesDecodedDelta: number | null) => { - if (this.active === slot && !slot.failed) { - this.events.onSfuDecodedFrameSample?.( - framesDecodedDelta, - slot.revision, - slot.mediaIdentity, - slot.connectionId, - ); - } - }, onFirstDecodedFrame: () => this.handlePendingDecodedFrame(slot), onState: (state: "connected" | "reconnecting") => { if (this.active === slot && !slot.failed) { @@ -492,7 +473,6 @@ export class ViewerSfuRoute { new SfuSubscriber(subscriberEvents); const preparedCandidate = this.route.getPreparedCandidate(); slot = { - mediaIdentity: `${publicationGeneration}:${++this.subscriberGeneration}`, connectionId: message.connectionId, revision: message.revision, phase, diff --git a/src/client/native/media-bridge.ts b/src/client/native/media-bridge.ts index 574e1ae..6d18132 100644 --- a/src/client/native/media-bridge.ts +++ b/src/client/native/media-bridge.ts @@ -15,6 +15,7 @@ const BRIDGE_TIMEOUT_MS = 8_000; const MAX_PENDING_CANDIDATES = 64; export class NativeMediaBridgeError extends Error {} +export class NativeMediaBridgeInitializationError extends NativeMediaBridgeError {} export interface NativeMediaBridgeControl { prepareLocalEdge( @@ -32,12 +33,13 @@ export interface NativeMediaBridgeControl { } export class NativeMediaBridge { - readonly stream = new MediaStream(); + readonly stream: MediaStream; readonly connectionId = createOpaqueId(); - private readonly peer = new RTCPeerConnection(); + private readonly peer: RTCPeerConnection; private readonly pendingCandidates: SignalCandidate[] = []; private unsubscribe: (() => void) | null = null; + private edgeRequested = false; private remoteDescriptionSet = false; private ready = false; private disposed = false; @@ -48,10 +50,18 @@ export class NativeMediaBridge { private readonly shareId: string, private readonly control: NativeMediaBridgeControl, private readonly onFailed: () => void, - private readonly expectedAudio = false, private readonly sourceConnectionId?: string, ) { - observeDebugConnection(this.peer, { connectionId: this.connectionId, sourceConnectionId, shareId, role: "native-preview" }); + let peer: RTCPeerConnection | undefined; + try { + this.stream = new MediaStream(); + this.peer = peer = new RTCPeerConnection(); + observeDebugConnection(this.peer, { connectionId: this.connectionId, sourceConnectionId, shareId, role: "native-preview" }); + } catch (error) { + peer?.close(); + debugError("native-bridge", "initialization-failed", error); + throw new NativeMediaBridgeInitializationError("Native media bridge could not initialize", { cause: error }); + } } collectMetrics(accumulator: StatsAccumulator): Promise { @@ -66,7 +76,7 @@ export class NativeMediaBridge { ); } - async start(): Promise { + async start(expectedAudio = false): Promise { if (this.disposed || this.unsubscribe) { throw new NativeMediaBridgeError("Native media bridge is unavailable"); } @@ -78,7 +88,7 @@ export class NativeMediaBridge { !this.ready && this.peer.connectionState === "connected" && this.stream.getVideoTracks().length > 0 && - (!this.expectedAudio || this.stream.getAudioTracks().length > 0) + (!expectedAudio || this.stream.getAudioTracks().length > 0) ) { this.ready = true; this.rejectStart = null; @@ -127,7 +137,7 @@ export class NativeMediaBridge { } : null, }, - ).catch(() => this.fail("candidate-signal")); + ).catch((error) => this.fail("candidate-signal", error)); }); this.unsubscribe = this.control.onEvent((event) => { if (event.shareId !== this.shareId) return; @@ -151,7 +161,7 @@ export class NativeMediaBridge { return; } void addRemoteIceCandidate(this.peer, event.candidate) - .catch(() => this.fail("candidate-apply")); + .catch((error) => this.fail("candidate-apply", error)); } else if ( event.type === "edge-state" && (event.state === "failed" || event.state === "closed") @@ -161,6 +171,7 @@ export class NativeMediaBridge { }); const negotiate = async () => { + this.edgeRequested = true; const offer = await this.control.prepareLocalEdge( this.shareId, this.connectionId, @@ -195,7 +206,7 @@ export class NativeMediaBridge { try { return await Promise.race([negotiate(), started]); } catch (error) { - this.fail("negotiation"); + this.fail("negotiation", error); throw error instanceof NativeMediaBridgeError ? error : new NativeMediaBridgeError("Native media bridge failed", { cause: error }); @@ -213,14 +224,18 @@ export class NativeMediaBridge { this.pendingCandidates.length = 0; this.peer.close(); this.stream.getTracks().forEach((track) => track.stop()); - void this.control.closeEdge(this.shareId, this.connectionId).catch( - () => undefined, - ); + if (this.edgeRequested) { + void this.control.closeEdge(this.shareId, this.connectionId).catch( + () => undefined, + ); + } } - private fail(reason: string): void { + private fail(reason: string, error?: unknown): void { if (this.disposed) return; - debugEvent("native-bridge", "failed", { type: reason, state: this.ready ? "active" : "starting" }); + const details = { type: reason, state: this.ready ? "active" : "starting" }; + if (error === undefined) debugEvent("native-bridge", "failed", details); + else debugError("native-bridge", "failed", error, details); void this.reportFailureDiagnostics(); if (this.ready) this.onFailed(); this.dispose(); diff --git a/src/client/native/native-viewer-peer.ts b/src/client/native/native-viewer-peer.ts index 3614a0d..83a5c53 100644 --- a/src/client/native/native-viewer-peer.ts +++ b/src/client/native/native-viewer-peer.ts @@ -487,11 +487,10 @@ class NativeViewerPeer implements ViewerMediaPeer { this.sessionId, this.client, () => this.handlePlaybackFailure(bridge, connectionId), - result.audio, connectionId, ); this.bridge = bridge; - void bridge.start().then((stream) => { + void bridge.start(result.audio).then((stream) => { if (this.disposed || this.bridge !== bridge || this.connectionId !== connectionId) { bridge.dispose(); return; diff --git a/src/client/pages/ControlsPreview.tsx b/src/client/pages/ControlsPreview.tsx index 6ffa0cd..ccbe2e4 100644 --- a/src/client/pages/ControlsPreview.tsx +++ b/src/client/pages/ControlsPreview.tsx @@ -44,6 +44,7 @@ export function ControlsPreview() { const [launchMode, setLaunchMode] = useState("link"); const [launchSite, setLaunchSite] = useState(""); const [launchPassword, setLaunchPassword] = useState(""); + const [launchLan, setLaunchLan] = useState("192.0.2.10"); const [preset, setPreset] = useState("1080p30"); const [policy, setPolicy] = useState<"open" | "private">("open"); const [roomPassword, setRoomPassword] = useState(false); @@ -208,6 +209,10 @@ export function ControlsPreview() {
event.preventDefault()} />
diff --git a/src/client/pages/HostPage.tsx b/src/client/pages/HostPage.tsx index 719c278..9dfa535 100644 --- a/src/client/pages/HostPage.tsx +++ b/src/client/pages/HostPage.tsx @@ -164,7 +164,7 @@ import { } from "../native/native-sender-peer"; import { NativeSfuPublisher } from "../native/native-sfu-publisher"; import { SfuPublisher } from "../sfu/publisher"; -import { NativeMediaBridge } from "../native/media-bridge"; +import { NativeMediaBridge, NativeMediaBridgeError } from "../native/media-bridge"; import { NativeMediaIngress } from "../native/media-ingress"; import { defaultNativeCapturePath, @@ -492,6 +492,7 @@ export function HostPage({ action: HostAction, target: NoticeValue["target"] = "operation", ): void { + if (error instanceof NativeMediaBridgeError) action = "connection"; const permissionMissing = isCapturePermissionFailure(error, action); setNoticeValue({ kind: "text", text: readableError(error, action), target, tone: permissionMissing ? "warn" : "bad", comic: permissionMissing ? "hint-capture-browser" : action === "connection" ? "route-failed" : action === "capture" || action === "source" @@ -1164,6 +1165,21 @@ export function HostPage({ await nativePreviewTailRef.current; if (!isCurrentShare(generation, shareGeneration)) return null; if (nativeClientRef.current !== client) throw new Error("Piik App is unavailable"); + // Acquire the required Browser peer before starting capture. Pending + // cancellation owns this bridge, but still closes control to abort startup. + bridge = new NativeMediaBridge( + shareGeneration, + client, + () => { + if ( + nativeMediaBridgeRef.current === bridge && + isCurrentShare(generation, shareGeneration) + ) { + endSharing({ key: "host.shareEnded" }, true, "source-failed", "bad"); + } + }, + ); + nativeMediaBridgeRef.current = bridge; nativeSourceAudioRef.current = undefined; nativeEventCleanup = client.onEvent((event) => { if (event.shareId !== shareGeneration || !isCurrentShare(generation, shareGeneration) || nativeClientRef.current !== client) return; @@ -1196,33 +1212,21 @@ export function HostPage({ }); shareStarted = true; if (!isCurrentShare(generation, shareGeneration)) { + if (nativeMediaBridgeRef.current === bridge) nativeMediaBridgeRef.current = null; + bridge.dispose(); await client.stopShare(shareGeneration).catch(() => discardNativeClient(client)); return null; } if (nativeClientRef.current !== client) throw new Error("Piik App is unavailable"); nativeSourceAudioRef.current ??= started.sourceAudio ?? started.audio; videoCodecRef.current = manualVideoCodecPreference(started.codec); - bridge = new NativeMediaBridge( - shareGeneration, - client, - () => { - if ( - nativeMediaBridgeRef.current === bridge && - isCurrentShare(generation, shareGeneration) - ) { - endSharing({ key: "host.shareEnded" }, true, "source-failed", "bad"); - } - }, - started.audio, - ); // Register ownership before waiting for the local bridge. A native edge // may fail immediately after becoming ready. ownNativeClient(client); nativeShareGenerationRef.current = shareGeneration; - nativeMediaBridgeRef.current = bridge; nativeModeRef.current = true; setNativeActive(true); - const stream = await bridge.start(); + const stream = await bridge.start(started.audio); if (!isCurrentShare(generation, shareGeneration)) { if (nativeMediaBridgeRef.current === bridge) disposeNativeShare(); else bridge.dispose(); @@ -1238,6 +1242,7 @@ export function HostPage({ ) { disposeNativeShare(); } else { + if (nativeMediaBridgeRef.current === bridge) nativeMediaBridgeRef.current = null; bridge?.dispose(); if (shareStarted) { nativeShareCleanupRef.current = client @@ -2858,6 +2863,7 @@ export function HostPage({ } function setCaptureError(error: unknown, source: BrowserCaptureSource | undefined, action: "source" | "capture") { + debugError("capture", "failed", error, { action, source }); const target = action === "source" ? "operation" : "television"; if (source !== "camera") { setNoticeError(error, action, target); diff --git a/src/client/pages/ViewerPage.tsx b/src/client/pages/ViewerPage.tsx index bdbe3c3..deb7765 100644 --- a/src/client/pages/ViewerPage.tsx +++ b/src/client/pages/ViewerPage.tsx @@ -63,7 +63,6 @@ import { P2pQualityProbe, type CandidateQualityProbeResult, } from "../media/candidate-quality-probe"; -import { DecodedFrameStallDetector } from "../media/decoded-frame-stall"; import type { QualitySettings } from "../media/quality"; import { relayCapacityMessageForBrowser } from "../media/relay-capability"; import { @@ -491,7 +490,6 @@ export function ViewerPage({ Extract > = []; let pendingPeer: PendingPeerRoute | null = null; - const decodedFrameStall = new DecodedFrameStallDetector(); const messageAuthority = new ViewerMessageAuthority(); const relayChildEvidenceStore = new ViewerQualityEvidenceStore((values) => { if (active) setRelayChildEvidence(values); @@ -543,9 +541,6 @@ export function ViewerPage({ } let pageSuspended = document.visibilityState !== "visible"; - const syncDecodedFrameStallPause = (): void => { - decodedFrameStall.setPaused(currentHostPaused || pageSuspended); - }; const suspendForPageLifecycle = (): void => { // Losing observation does not invalidate media already proved playable. invalidateQualityPresentation(); @@ -555,7 +550,6 @@ export function ViewerPage({ viewerSfuRoute?.resetQualityProbe(); pendingPeer?.qualityProbe?.reset(); if (pendingPeer) pendingPeer.qualityResult = "pending"; - syncDecodedFrameStallPause(); if (newlySuspended) { invalidateSenderQualityEvidence(); if (currentRoutePolicy.topologyOptimization) { @@ -567,9 +561,6 @@ export function ViewerPage({ invalidateQualityPresentation(); rearmCurrentFrameProof(); pageSuspended = document.visibilityState !== "visible"; - syncDecodedFrameStallPause(); - if (pageSuspended) return; - decodedFrameStall.rebaseline(); }; const handleVisibilityChange = (): void => { if (document.visibilityState === "visible") { @@ -578,7 +569,6 @@ export function ViewerPage({ suspendForPageLifecycle(); } }; - syncDecodedFrameStallPause(); document.addEventListener("freeze", suspendForPageLifecycle); document.addEventListener("resume", recoverFromPageLifecycle); document.addEventListener("visibilitychange", handleVisibilityChange); @@ -601,7 +591,6 @@ export function ViewerPage({ { onStatus: (status) => { if (active) { - if (status !== "connected") decodedFrameStall.allowReportRetry(); dispatchPresentation({ type: "signal", signal: status }); } }, @@ -922,32 +911,6 @@ export function ViewerPage({ ); } - function observeActiveDecodedFrames( - route: "peer" | "sfu", - identity: string, - framesDecodedDelta: number | null, - authorityRevision: number, - connectionId: string, - ): void { - if ( - currentHostPaused || - !decodedFrameStall.observe(`${route}:${identity}`, framesDecodedDelta) - ) { - return; - } - if (authorityRevision !== currentRouteRevision) { - decodedFrameStall.allowReportRetry(); - return; - } - const sent = signal.send({ - type: "route-failed", - revision: authorityRevision, - phase: "active", - connectionId, - }); - if (!sent) decodedFrameStall.allowReportRetry(); - } - function pendingPeerHasDecodedFrame( probe: PendingPeerRoute, ): probe is PendingPeerRoute & { peer: ViewerMediaPeer; snapshot: PeerSnapshot } { @@ -1118,22 +1081,6 @@ export function ViewerPage({ ); reconcileRelayChildren(previousChildPeerIds, revision); }, - onSfuDecodedFrameSample: ( - framesDecodedDelta, - revision, - mediaIdentity, - connectionId, - ) => { - if (active && viewerSfuRoute === route) { - observeActiveDecodedFrames( - "sfu", - mediaIdentity, - framesDecodedDelta, - revision, - connectionId, - ); - } - }, onSfuUpdate: (metrics, revision) => { if (active && viewerSfuRoute === route) { if (metrics) { @@ -1386,13 +1333,6 @@ export function ViewerPage({ provePendingPeer(); } else if (active && peerRef.current === peer) { activePeerMetrics = snapshot.metrics; - observeActiveDecodedFrames( - "peer", - `${probe.parentPeerId}:${snapshot.connectionId}`, - snapshot.metrics.intervalFramesDecoded, - currentRouteRevision, - snapshot.connectionId, - ); offerPeerQualityEvidence(snapshot, peer); setPeerSnapshot(snapshot); dispatchPresentation({ @@ -1481,13 +1421,6 @@ export function ViewerPage({ onUpdate: (snapshot) => { if (active) { activePeerMetrics = snapshot.metrics; - observeActiveDecodedFrames( - "peer", - `${snapshot.peerId}:${snapshot.connectionId}`, - snapshot.metrics.intervalFramesDecoded, - currentRouteRevision, - snapshot.connectionId, - ); offerPeerQualityEvidence(snapshot, peer); if ( !currentHostOnline && @@ -1579,7 +1512,6 @@ export function ViewerPage({ invalidatePresentedMedia(); } currentHostPaused = sharingPaused; - syncDecodedFrameStallPause(); dispatchPresentation({ type: "host", host: sharingPaused @@ -1800,7 +1732,6 @@ export function ViewerPage({ invalidatePresentedMedia(); } currentHostPaused = message.paused; - syncDecodedFrameStallPause(); dispatchPresentation({ type: "host", host: message.paused @@ -1830,8 +1761,6 @@ export function ViewerPage({ setAssignedRoute(null); currentHostOnline = false; currentHostPaused = false; - decodedFrameStall.reset(); - syncDecodedFrameStallPause(); clearViewerSfuRoute(); clearPeerState(true); clearHostPresence(); diff --git a/src/client/pages/host-page-notices.ts b/src/client/pages/host-page-notices.ts index aef4ade..c18aa46 100644 --- a/src/client/pages/host-page-notices.ts +++ b/src/client/pages/host-page-notices.ts @@ -1,5 +1,5 @@ import type { ServerMessage } from "../../shared/protocol"; -import { NativeMediaBridgeError } from "../native/media-bridge"; +import { NativeMediaBridgeError, NativeMediaBridgeInitializationError } from "../native/media-bridge"; import { currentLang, joinSentences, say, type CopyKey } from "../ui/copy"; export type HostAction = @@ -44,7 +44,8 @@ export function hostActionErrorNotice( error: unknown, action: HostAction, ): string { - if (error instanceof NativeMediaBridgeError) return say(HOST_ACTION_FALLBACK.connection); + if (error instanceof NativeMediaBridgeInitializationError) return say("native.fail.browserMedia"); + if (error instanceof NativeMediaBridgeError) return say("native.fail.edge"); if (isCapturePermissionFailure(error, action)) return say("host.capture.cancelled"); if ( error instanceof DOMException && diff --git a/src/client/sfu/subscriber.ts b/src/client/sfu/subscriber.ts index 3e766e4..1f2a533 100644 --- a/src/client/sfu/subscriber.ts +++ b/src/client/sfu/subscriber.ts @@ -15,7 +15,6 @@ interface SubscriberEvents { onStream: (stream: MediaStream | null) => void; onVideoAvailability?: (available: boolean) => void; onStats?: (metrics: ConnectionMetrics) => void; - onDecodedFrameSample?: (framesDecodedDelta: number | null) => void; onFirstDecodedFrame?: () => boolean; onState?: (state: "connected" | "reconnecting") => void; onDisconnected?: () => void; @@ -228,10 +227,7 @@ export class SfuSubscriber { const video = this.video; const stats = this.stats; if (!peer || !this.active) return; - if (!video || this.statsInFlight === stats) { - this.events.onDecodedFrameSample?.(null); - return; - } + if (!video || this.statsInFlight === stats) return; this.statsInFlight = stats; try { const report = await peer.pc.getStats(); @@ -243,12 +239,9 @@ export class SfuSubscriber { "receive", stats, ); - this.events.onDecodedFrameSample?.(metrics.intervalFramesDecoded); this.events.onStats?.(metrics); } catch (error) { debugRtcFailure(peer.pc, error); - if (this.peer === peer && this.stats === stats) - this.events.onDecodedFrameSample?.(null); } finally { if (this.statsInFlight === stats) this.statsInFlight = null; } diff --git a/src/client/styles.css b/src/client/styles.css index a6a7b23..a230d24 100644 --- a/src/client/styles.css +++ b/src/client/styles.css @@ -116,6 +116,7 @@ ul { margin: 0; padding: 0; list-style: none; } .lr-brand { display: grid; + flex: none; width: 44px; height: 44px; place-items: center; border: 0; @@ -128,6 +129,7 @@ ul { margin: 0; padding: 0; list-style: none; } .lr-brand:active { transform: scale(.92); } .lr-top-right { display: inline-flex; gap: 10px; align-items: center; } +.lr-top > .lr-top-right { min-width: 0; flex-wrap: wrap; justify-content: flex-end; } /* LED state strip */ .lr-leds { @@ -915,8 +917,10 @@ html[data-theme="dark"] .lr-invite-url { background: #1a2634; border-color: #334 and the zoom never returns. Pointer devices keep the dense line. */ @media (pointer: coarse) { .lr-invite-url { font-size: 16px; } } /* LCD room code */ +.lr-room-chip { display: inline-flex; align-items: center; gap: 6px; max-width: 100%; } .lr-lcd { display: inline-flex; + flex: none; align-items: center; gap: 10px; padding: 8px 12px; @@ -1532,7 +1536,7 @@ html[data-theme="dark"] .lr-door-glyph { color: #7d92a8; } .lr-client-lan legend { padding: 0; font-size: 14px; font-weight: 750; } .lr-client-lan-hint { margin: 5px 0 10px; color: var(--muted); font-size: 12px; } .lr-client-lan-options { display: grid; gap: 8px; max-height: 192px; overflow-y: auto; padding: 3px; } -.lr-client-lan-option { width: 100%; min-width: 0; justify-content: flex-start; border-radius: var(--control-radius); cursor: pointer; } +.lr-client-lan-option { width: 100%; min-width: 0; height: auto; min-height: 52px; grid-template-columns: auto minmax(0, 1fr); justify-content: stretch; white-space: normal; border-radius: var(--control-radius); cursor: pointer; } .lr-client-lan-option input { flex: 0 0 auto; width: 18px; height: 18px; margin: 0; accent-color: var(--action); } .lr-client-lan-option span { display: grid; min-width: 0; gap: 3px; text-align: left; } .lr-client-lan-option strong { font-size: 13px; overflow-wrap: anywhere; } @@ -1750,7 +1754,9 @@ html[data-theme="dark"] .lr-switch::after { background: var(--ink); } } .lr-group-actions .lr-btn { flex: 1 1 0; min-width: 0; min-height: 52px; padding: 4px 5px; gap: 6px; } .lr-group-actions .lr-btn .lr-cap { white-space: normal; text-align: center; line-height: 1.25; font-size: 11px; } - .lr-lcd { flex: 1; justify-content: center; } + .lr-lcd { gap: 6px; padding-inline: 8px; font-size: 22px; letter-spacing: .2em; } + .lr-room-chip .lr-btn { min-width: 44px; width: 44px; height: 44px; padding: 0; } + .lr-room-chip .lr-cap { display: none; } .lr-tiles { display: flex; width: 100%; gap: 8px; } .lr-tiles .lr-tile { flex: 1 1 0; min-width: 0; padding: 10px 6px 8px; } .lr-meter { display: grid; grid-template-columns: repeat(2, minmax(0, 1fr)); width: 100%; } @@ -1773,6 +1779,9 @@ html[data-theme="dark"] .lr-switch::after { background: var(--ink); } .lr-source-option { min-height: 78px; padding: 6px 8px; } .lr-source-option-icon { width: 34px; height: 34px; } .lr-leds-label { display: none; } + .lr-top-right { gap: 8px; } + .lr-leds { gap: 4px; padding: 6px; } + .lr-header-controls .lr-btn { width: 40px; min-width: 40px; padding: 0; } .lr-header-controls .lr-btn .lr-cap { display: none; } } @@ -1879,11 +1888,10 @@ html[data-theme="dark"] .lr-switch::after { background: var(--ink); } } } @media (max-width: 360px) { + .lr-lcd { gap: 4px; padding-inline: 6px; font-size: 18px; } + .lr-room-chip .lr-btn { min-width: 36px; width: 36px; } .lr-top { gap: 6px; padding-inline: 10px; } - .lr-brand { flex-shrink: 0; } .lr-top-right { gap: 6px; } - .lr-top > .lr-top-right { min-width: 0; flex-wrap: wrap; justify-content: flex-end; } - .lr-leds { gap: 4px; padding: 6px; } .lr-leds i { width: 8px; height: 8px; } .lr-lang { padding: 2px; border-width: 2px; } .lr-lang::before { top: 2px; bottom: 2px; left: 2px; width: calc((100% - 4px) / var(--lang-count)); } @@ -1946,7 +1954,7 @@ svg[data-comic-motion="still"] .lr-person-eyes > ellipse { overflow: visible; transform: translate(var(--tooltip-offset-x, 0px), var(--tooltip-offset, 4px)); width: max-content; - max-width: 320px; + max-width: min(320px, var(--tooltip-max-width, calc(100vw - 16px))); z-index: 30; background: var(--paper); border: 2.5px solid var(--comic-tone, var(--ink)); @@ -1960,7 +1968,7 @@ svg[data-comic-motion="still"] .lr-person-eyes > ellipse { .lr-comic-tip * { animation-play-state: paused !important; } .lr-comic-tip.is-text { padding: 10px 14px; - max-width: min(320px, calc(100vw - 16px)); + max-width: min(320px, var(--tooltip-max-width, calc(100vw - 16px))); color: var(--ink); font: 600 14px/1.6 var(--font); text-align: start; @@ -1969,7 +1977,14 @@ svg[data-comic-motion="still"] .lr-person-eyes > ellipse { text-wrap: pretty; } .lr-comic-tip.has-comic.is-text { width: 252px; } -.lr-comic-tip > svg { +.lr-comic-tip-content { + display: block; + max-height: max(0px, calc(var(--tooltip-max-height, 100dvh - 16px) - 25px)); + overflow: auto; + overscroll-behavior: contain; + scrollbar-width: thin; +} +.lr-comic-tip-content > svg { display: block; max-width: 100%; height: auto; @@ -2071,13 +2086,13 @@ svg[data-comic-motion="still"] .lr-person-eyes > ellipse { @media (max-width: 640px) { .lr-group-actions .lr-comic-tip-wrap { flex: 1 1 0; min-width: 0; } .lr-group-actions .lr-comic-tip-wrap .lr-btn { width: 100%; } - .lr-comic-tip { max-width: min(320px, 78vw); } + .lr-comic-tip { max-width: min(320px, 78vw, var(--tooltip-max-width, calc(100vw - 16px))); } } .lr-comic-tip-caption { display: block; } -.lr-comic-tip > svg + .lr-comic-tip-caption { margin-top: 8px; } +.lr-comic-tip-content > svg + .lr-comic-tip-caption { margin-top: 8px; } /* ---------- reduced motion: everything collapses to instant ---------- */ @@ -2129,7 +2144,7 @@ html[data-theme="dark"] .lr-tile:not(.is-selected) small { color: #8ba0b4; } @media (max-width: 640px) { - .lr-tv-screen .lr-comic-tip { max-width: 62vw; } + .lr-tv-screen .lr-comic-tip { max-width: min(62vw, var(--tooltip-max-width, calc(100vw - 16px))); } } /* ---------- glyph draw-in (一笔一划) ---------- */ diff --git a/src/client/webrtc/host-peer.ts b/src/client/webrtc/host-peer.ts index 7a8564a..ffe4df6 100644 --- a/src/client/webrtc/host-peer.ts +++ b/src/client/webrtc/host-peer.ts @@ -182,7 +182,7 @@ export class HostPeer { this.videoSender = videoTransceiver.sender; if (this.encodedStreamsEnabled) { this.encodedOutput = new BrowserEncodingOutput(this.videoSender, () => { - if (!this.disposed) { this.dispose(); this.emit(); } + this.fail(); }, { connectionId: this.connectionId, peerId: this.peerId }); } this.attachVideoPool(this.stream.getVideoTracks()[0]!); @@ -194,9 +194,7 @@ export class HostPeer { if (this.encodedStreamsEnabled) { // The connection flag also owns audio, including a later source with audio. const audio = encodedStreams(this.audioSender); - void audio.readable.pipeTo(audio.writable).catch(() => { - if (!this.disposed) { this.dispose(); this.emit(); } - }); + void audio.readable.pipeTo(audio.writable).catch((error) => this.fail(error)); } await this.enqueueSenderMutation(async () => { if (this.disposed || !this.videoSender || !this.audioSender) { @@ -275,7 +273,7 @@ export class HostPeer { audioTransceiver.direction = previousAudioDirection; } if (videoRollback.status === "rejected") { - this.dispose(); + this.fail(videoRollback.reason); } this.setError(error, "host.fail.source"); return false; @@ -589,7 +587,7 @@ export class HostPeer { return true; } catch (error) { if (!this.disposed && sender.track !== previous) { - try { await this.waitForOperation(() => sender.replaceTrack(previous)); } catch { this.dispose(); } + try { await this.waitForOperation(() => sender.replaceTrack(previous)); } catch (rollbackError) { this.fail(rollbackError); } } if (next !== previous && next !== this.encodedOutput?.track) next.stop(); if (this.disposed) return false; @@ -598,7 +596,7 @@ export class HostPeer { } }), requestKey, - () => { if (owns()) { this.dispose(); this.emit(); } }); + () => { if (owns()) this.fail(); }); this.pooledVideo = binding; if (!binding) this.encodedOutput.passthrough(requestKey); binding?.setPaused(this.paused); @@ -818,6 +816,16 @@ export class HostPeer { this.emit(); } + private fail(error?: unknown): void { + if (this.disposed) return; + debugError("webrtc", "sender-media-failed", error, { connectionId: this.connectionId }); + this.snapshot = { ...this.snapshot, connectionState: "failed", error: { key: "host.fail.connection" } }; + this.dispose(); + // Expected disposal is silent; terminal media failure must still reach + // its preparation/recovery owner after owned resources have retired. + this.events.onUpdate(this.getSnapshot()); + } + private async configureSender( sender: RTCRtpSender, audioSender?: RTCRtpSender, diff --git a/tests/decoded-frame-stall.test.ts b/tests/decoded-frame-stall.test.ts deleted file mode 100644 index 92d48e2..0000000 --- a/tests/decoded-frame-stall.test.ts +++ /dev/null @@ -1,102 +0,0 @@ -import { describe, expect, it } from "vitest"; - -import { - DecodedFrameStallDetector, - ROUTE_DECODED_FRAME_STALL_MS, -} from "../src/client/media/decoded-frame-stall.ts"; - -describe("DecodedFrameStallDetector", () => { - it("reports one exact active stall and resets on progress or identity change", () => { - const detector = new DecodedFrameStallDetector(); - expect(detector.observe("peer:a", 0, 0)).toBe(false); - expect( - detector.observe("peer:a", 0, ROUTE_DECODED_FRAME_STALL_MS), - ).toBe(true); - expect( - detector.observe("peer:a", 0, ROUTE_DECODED_FRAME_STALL_MS + 1), - ).toBe(false); - expect(detector.observe("peer:a", 1, ROUTE_DECODED_FRAME_STALL_MS + 2)).toBe( - false, - ); - expect( - detector.observe("peer:a", 0, ROUTE_DECODED_FRAME_STALL_MS * 2 + 2), - ).toBe(true); - expect(detector.observe("sfu:b", 0, ROUTE_DECODED_FRAME_STALL_MS * 3)).toBe( - false, - ); - }); - - it("does not turn unknown samples into elapsed stall evidence", () => { - const detector = new DecodedFrameStallDetector(); - expect(detector.observe("peer:a", 0, 0)).toBe(false); - expect( - detector.observe("peer:a", null, ROUTE_DECODED_FRAME_STALL_MS), - ).toBe(false); - expect( - detector.observe( - "peer:a", - 0, - ROUTE_DECODED_FRAME_STALL_MS * 2 - 1, - ), - ).toBe(false); - expect( - detector.observe("peer:a", 0, ROUTE_DECODED_FRAME_STALL_MS * 2), - ).toBe(true); - }); - - it("suppresses and rebases progress while paused", () => { - const detector = new DecodedFrameStallDetector(); - detector.observe("peer:1:a", 0, 0); - detector.setPaused(true, ROUTE_DECODED_FRAME_STALL_MS); - expect( - detector.observe("peer:1:a", 0, ROUTE_DECODED_FRAME_STALL_MS * 3), - ).toBe(false); - detector.setPaused(false, ROUTE_DECODED_FRAME_STALL_MS * 3); - expect( - detector.observe("peer:1:a", 0, ROUTE_DECODED_FRAME_STALL_MS * 4 - 1), - ).toBe(false); - expect( - detector.observe("peer:1:a", 0, ROUTE_DECODED_FRAME_STALL_MS * 4), - ).toBe(true); - }); - - it("rebases elapsed time after the page resumes", () => { - const detector = new DecodedFrameStallDetector(); - detector.observe("peer:1:a", 0, 0); - detector.rebaseline(ROUTE_DECODED_FRAME_STALL_MS * 4); - - expect( - detector.observe( - "peer:1:a", - null, - ROUTE_DECODED_FRAME_STALL_MS * 5 - 1, - ), - ).toBe(false); - expect( - detector.observe( - "peer:1:a", - 0, - ROUTE_DECODED_FRAME_STALL_MS * 5, - ), - ).toBe(false); - expect( - detector.observe( - "peer:1:a", - 0, - ROUTE_DECODED_FRAME_STALL_MS * 6 - 1, - ), - ).toBe(true); - }); - - it("retries an exact stall when its failure message was not sent", () => { - const detector = new DecodedFrameStallDetector(); - expect(detector.observe("peer:1:a", 0, 0)).toBe(false); - expect( - detector.observe("peer:1:a", 0, ROUTE_DECODED_FRAME_STALL_MS), - ).toBe(true); - detector.allowReportRetry(); - expect( - detector.observe("peer:1:a", 0, ROUTE_DECODED_FRAME_STALL_MS + 1), - ).toBe(true); - }); -}); diff --git a/tests/host-page-notice.test.ts b/tests/host-page-notice.test.ts index d2d6f72..b56de01 100644 --- a/tests/host-page-notice.test.ts +++ b/tests/host-page-notice.test.ts @@ -1,7 +1,7 @@ import { describe, expect, it } from "vitest"; import { setCopy } from "../src/client/ui/copy.ts"; -import { NativeMediaBridgeError } from "../src/client/native/media-bridge"; +import { NativeMediaBridgeError, NativeMediaBridgeInitializationError } from "../src/client/native/media-bridge"; import { hostActionErrorNotice, hostServerErrorNotice, @@ -12,9 +12,24 @@ import { setCopy({ lang: "zh" }); describe("host error notices", () => { + it.each([ + ["zh", "浏览器无法建立媒体连接。请刷新页面，或换一个浏览器重试。"], + ["en", "The browser could not create a media connection. Refresh the page or try another browser."], + ] as const)("identifies Browser initialization failure in %s without blaming capture permission", (lang, notice) => { + setCopy({ lang }); + try { + const failure = new NativeMediaBridgeInitializationError("private detail", { + cause: new DOMException("private cause", "NotAllowedError"), + }); + expect(hostActionErrorNotice(failure, "capture")).toBe(notice); + } finally { + setCopy({ lang: "zh" }); + } + }); + it("classifies the native preview bridge as a connection failure", () => { expect(hostActionErrorNotice(new NativeMediaBridgeError("private detail"), "capture")) - .toBe(hostActionErrorNotice(new Error("private detail"), "connection")); + .toBe("Piik App 媒体连接失败"); }); it("maps every server error code without exposing server text", () => { diff --git a/tests/host-peer-tracks.test.ts b/tests/host-peer-tracks.test.ts index 5564795..5511b50 100644 --- a/tests/host-peer-tracks.test.ts +++ b/tests/host-peer-tracks.test.ts @@ -19,6 +19,8 @@ import { type ViewerRelayPeerFactory, } from "../src/client/webrtc/viewer-relay.ts"; import { setCopy } from "../src/client/ui/copy.ts"; +import * as encodingOutput from "../src/client/media/browser-encoding-output"; +import type { BrowserEncodingPool } from "../src/client/media/browser-encoding-pool"; import type { IceConfig, ParticipantRouteAssignment, @@ -527,6 +529,84 @@ afterEach(() => { vi.unstubAllGlobals(); }); +function encodedPeerFixture() { + let failVideo!: () => void; + let audio!: ReadableStreamDefaultController; + const output = { track: createTrack("video", "carrier"), dispose: vi.fn(), passthrough: vi.fn(), setPaused: vi.fn() }; + const binding = { dispose: vi.fn(), carrierScale: vi.fn<() => number | undefined>(() => undefined), + setPaused: vi.fn(), updateProfile: vi.fn(async () => undefined), metrics: (value: unknown) => value }; + const create = vi.fn(() => binding as never); + const pool = { create } as unknown as BrowserEncodingPool; + vi.spyOn(encodingOutput, "supportsBrowserEncoding").mockReturnValue(true); + vi.spyOn(encodingOutput, "BrowserEncodingOutput").mockImplementation(function (_sender, onFailure) { + failVideo = onFailure; + return output as unknown as encodingOutput.BrowserEncodingOutput; + }); + vi.spyOn(encodingOutput, "encodedStreams").mockReturnValue({ + readable: new ReadableStream({ start(controller) { audio = controller; } }), + writable: new WritableStream(), + } as never); + return { pool, create, binding, output, failVideo: () => failVideo(), + failAudio: () => audio.error(new Error("audio transform failed")) }; +} + +describe("HostPeer terminal media failure", () => { + it.each(["video", "audio", "fallback", "rollback"])("reports %s failure once and keeps borrowed source tracks alive", async (kind) => { + const fixture = encodedPeerFixture(); + const source = createTrack("video", "source"), audio = createTrack("audio", "audio"); + const onUpdate = vi.fn(); + const peer = new HostPeer("child", { iceServers: [] }, createStream(source, audio), QUALITY_PROFILES["720p30"], + { sendSignal: () => true, onUpdate }, VP8_ONLY_VIDEO_CODEC, "exact-connection", false, fixture.pool); + await peer.start(); + const connection = FakePeerConnection.latest!, owned = connection.senders[0]!.track!; + onUpdate.mockClear(); + if (kind === "video") fixture.failVideo(); + else if (kind === "audio") fixture.failAudio(); + else if (kind === "fallback") fixture.create.mock.calls[0]![7](); + else { + fixture.binding.carrierScale.mockReturnValue(1); + connection.senders[0]!.failNextSetParameters = true; + connection.senders[0]!.replaceTrack.mockResolvedValueOnce(undefined).mockRejectedValueOnce(new Error("rollback failed")); + // Reflect that the first attachment succeeded before configuration failed. + connection.senders[0]!.track = fixture.output.track; + await fixture.create.mock.calls[0]![5](); + } + await vi.waitFor(() => expect(onUpdate).toHaveBeenCalledOnce()); + expect(onUpdate.mock.calls[0]![0]).toMatchObject({ peerId: "child", connectionId: "exact-connection", connectionState: "failed" }); + expect(peer.getSnapshot().connectionState).toBe("failed"); + expect(connection.connectionState).toBe("closed"); + expect(owned.stop).toHaveBeenCalledOnce(); + expect(source.stop).not.toHaveBeenCalled(); + expect(audio.stop).not.toHaveBeenCalled(); + fixture.failVideo(); + fixture.create.mock.calls[0]![7](); + expect(onUpdate).toHaveBeenCalledOnce(); + }); + + it("keeps expected disposal and late failures silent", async () => { + const fixture = encodedPeerFixture(), onUpdate = vi.fn(); + const peer = new HostPeer("child", { iceServers: [] }, createStream(createTrack("video", "source"), null), + QUALITY_PROFILES["720p30"], { sendSignal: () => true, onUpdate }, VP8_ONLY_VIDEO_CODEC, undefined, false, fixture.pool); + await peer.start(); + onUpdate.mockClear(); + peer.dispose(); + fixture.failVideo(); + fixture.failAudio(); + await new Promise((resolve) => setImmediate(resolve)); + expect(onUpdate).not.toHaveBeenCalled(); + }); + + it("reports the exact prepared child immediately instead of waiting for the route deadline", async () => { + const fixture = encodedPeerFixture(), onPreparedChildFailed = vi.fn(); + const owner = new HostProvisionalChild({ sendSignal: () => true, onPreparedChildFailed }); + owner.prepare({ ...hostProvisionalInput(7, ["candidate-child"], createStream(createTrack("video", "source"), null)), videoPool: fixture.pool }); + await vi.waitFor(() => expect(FakePeerConnection.latest!.localDescription).not.toBeNull()); + fixture.failVideo(); + expect(onPreparedChildFailed).toHaveBeenCalledExactlyOnceWith(7, "candidate-connection-7"); + owner.discard(); + }); +}); + describe("HostPeer source replacement", () => { it("does not expose WebRTC exception text in connection errors", async () => { const updates: PeerSnapshot[] = []; diff --git a/tests/host-quality-ownership.test.ts b/tests/host-quality-ownership.test.ts index 00d6ed3..0cb8e7b 100644 --- a/tests/host-quality-ownership.test.ts +++ b/tests/host-quality-ownership.test.ts @@ -9,6 +9,7 @@ import { } from "../src/client/media/quality"; import { NativeSenderPeer } from "../src/client/native/native-sender-peer"; import { NativeCompatibilityError } from "../src/client/native/client"; +import { NativeMediaBridgeError } from "../src/client/native/media-bridge"; import { reconcileBoundedMediaChildren } from "../src/client/webrtc/media-assignment"; import { debugError, debugEvent, debugOperation } from "../src/client/lib/debug"; import { hostActionErrorNotice, isCapturePermissionFailure } from "../src/client/pages/host-page-notices"; @@ -55,6 +56,7 @@ function fixture(launchedByClient = true) { let physical: QualitySettings = original; const track = { stop: vi.fn() }; const stream = { getTracks: () => [track] }; + const bridge = { start: vi.fn(async (_audio: boolean) => stream), dispose: vi.fn() }; const peer = { updateCaptureProfile: vi.fn(async () => true), updateProfile: vi.fn(async () => true), isConnected: vi.fn(() => true) }; const client = { updateShare: vi.fn(async () => undefined), replaceShareSource: vi.fn(async () => undefined), @@ -66,7 +68,7 @@ function fixture(launchedByClient = true) { stopReceive: vi.fn(async () => undefined), stopShare: vi.fn(async () => undefined) }; const route = { updateProfile: vi.fn(async () => true), resyncAuthoritative: vi.fn(async (): Promise => undefined) }; const state = { - debugError, debugEvent, debugOperation, NativeCompatibilityError, isCapturePermissionFailure, DOMException, + debugError, debugEvent, debugOperation, NativeCompatibilityError, NativeMediaBridgeError, isCapturePermissionFailure, DOMException, launchedByClient, NativeClient: { connect: vi.fn(async (): Promise => null) }, nativeClientConnectRef: ref | null>(null), ownNativeClient: vi.fn(), setJoiningRoom: vi.fn(), startSharing: vi.fn(), openCaptureSourcePicker: vi.fn(), @@ -81,7 +83,7 @@ function fixture(launchedByClient = true) { setCopiedRoomLink: vi.fn(), setPhase: vi.fn(), roomInitializationRef: ref(Promise.resolve()), roomRef: ref(null), createRoom: vi.fn(async () => { throw new Error("must not create an empty room"); }), readPreferredRoomId: () => null, disposeResources: vi.fn(), ApiError: class extends Error {}, - NativeMediaBridge: vi.fn(), manualVideoCodecPreference: vi.fn(), + NativeMediaBridge: vi.fn(function () { return bridge; }), manualVideoCodecPreference: vi.fn(), phase: "live", qualitySettingsRef: ref(original), advancedQualityRef: ref(original), qualityChangeRef: ref(null), pendingQualityChangeRef: ref(null), activeGenerationRef: ref(1), streamRef: ref(stream), sourceSwitchRef: ref(null), @@ -124,7 +126,7 @@ function fixture(launchedByClient = true) { context.startSharing = state.startSharing; context.startPeer = state.startPeer; context.openCaptureSourcePicker = state.openCaptureSourcePicker; - return { ...state, peer, client, route, track, physical: () => physical, + return { ...state, peer, client, route, track, bridge, physical: () => physical, context, startChild, openPicker: openPicker as () => Promise, closePicker: context.closeCaptureSourcePicker as () => void, releaseUnused: context.releaseUnusedNativeClient as () => void, @@ -436,6 +438,76 @@ describe("Host quality ownership", () => { expect(current.track.stop).not.toHaveBeenCalled(); }); + it("rejects unavailable Browser media before starting native capture or creating a room", async () => { + const current = fixture(); + const failure = new NativeMediaBridgeError("Preview unavailable", { + cause: new DOMException("Peer connection rejected", "NotAllowedError"), + }); + const report = vi.fn(); + Object.assign(current.context, { phase: "idle", debugError: report }); + current.activeGenerationRef.current = null; + current.nativeShareGenerationRef.current = null; + current.NativeMediaBridge.mockImplementationOnce(function () { throw failure; }); + await current.start(); + + expect(current.client.startShare).not.toHaveBeenCalled(); + expect(current.client.stopShare).not.toHaveBeenCalled(); + expect(current.client.close).toHaveBeenCalledOnce(); + expect(current.createRoom).not.toHaveBeenCalled(); + expect(current.roomMutationRef.current).toBeNull(); + expect(current.setPhase).toHaveBeenLastCalledWith("error"); + expect(current.setNoticeValue).toHaveBeenLastCalledWith(expect.objectContaining({ + target: "television", comic: "route-failed", tone: "bad", + })); + expect(report).toHaveBeenCalledWith("capture", "failed", failure, { action: "capture", source: undefined }); + }); + + it.each(["reject", "cancel", "accept"])("owns the Browser reservation while native startup is pending: %s", async (outcome) => { + const current = fixture(); + current.nativeShareGenerationRef.current = null; + const started = deferred<{ audio: boolean; codec: "h264" }>(); + current.client.startShare.mockReturnValue(started.promise); + const starting = current.startNative(); + const result = outcome === "accept" ? expect(starting).resolves.toBeDefined() + : expect(starting).rejects.toThrow("capture unavailable"); + await vi.waitFor(() => expect(current.client.startShare).toHaveBeenCalledOnce()); + expect(current.nativeMediaBridgeRef.current).toBe(current.bridge); + expect(current.nativeShareGenerationRef.current).toBeNull(); + expect(current.nativeModeRef.current).toBe(false); + expect(current.bridge.start).not.toHaveBeenCalled(); + if (outcome === "accept") { + // The App can supply its mixed audio track even when source audio was off. + started.resolve({ audio: true, codec: "h264" }); + } else { + if (outcome === "cancel") { + current.activeGenerationRef.current = null; + current.disposeNative(); + expect(current.bridge.dispose).toHaveBeenCalledOnce(); + expect(current.client.close).toHaveBeenCalledOnce(); + } + started.reject(new Error("capture unavailable")); + } + await result; + if (outcome === "accept") { + expect(current.bridge.start).toHaveBeenCalledExactlyOnceWith(true); + expect(current.nativeShareGenerationRef.current).toBe("share"); + expect(current.bridge.dispose).not.toHaveBeenCalled(); + } else { + expect(current.bridge.dispose).toHaveBeenCalled(); + expect(current.nativeMediaBridgeRef.current).toBeNull(); + expect(current.client.stopShare).not.toHaveBeenCalled(); + } + }); + + it.each(["camera", "browser"])("retains the %s capture cause when presenting its notice", (source) => { + const current = fixture(); + const report = vi.fn(); + current.context.debugError = report; + const cause = new DOMException("Device unavailable", "NotReadableError"); + current.context.setCaptureError(cause, source, "capture"); + expect(report).toHaveBeenCalledWith("capture", "failed", cause, { action: "capture", source }); + }); + it.each(["success", "replace-failed", "stopped"])("retires the whole microphone operation after stream replacement: %s", async outcome => { const current = fixture(); const permission = deferred(); @@ -759,7 +831,7 @@ describe("Host quality ownership", () => { expect(current.roomMutationRef.current).toBeNull(); }); - it("retires a cancelled start ACK before publishing audio, codec or bridge state", async () => { + it("retires a cancelled start ACK before publishing audio, codec or connecting the bridge", async () => { const current = fixture(); current.nativeShareGenerationRef.current = null; const started = deferred<{ audio: boolean; codec: "h264" }>(); @@ -772,7 +844,10 @@ describe("Host quality ownership", () => { expect(current.client.stopShare).toHaveBeenCalledWith("share"); expect(current.nativeSourceAudioRef.current).toBeUndefined(); expect(current.manualVideoCodecPreference).not.toHaveBeenCalled(); - expect(current.NativeMediaBridge).not.toHaveBeenCalled(); + expect(current.NativeMediaBridge).toHaveBeenCalledOnce(); + expect(current.bridge.start).not.toHaveBeenCalled(); + expect(current.bridge.dispose).toHaveBeenCalledOnce(); + expect(current.nativeMediaBridgeRef.current).toBeNull(); expect(current.setNativeActive).not.toHaveBeenCalled(); }); diff --git a/tests/native-media-bridge.test.ts b/tests/native-media-bridge.test.ts index b745098..dabb9c1 100644 --- a/tests/native-media-bridge.test.ts +++ b/tests/native-media-bridge.test.ts @@ -3,6 +3,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import { NativeMediaBridge, NativeMediaBridgeError, + NativeMediaBridgeInitializationError, type NativeMediaBridgeControl, } from "../src/client/native/media-bridge"; import type { NativeClientEvent } from "../src/client/native/wire"; @@ -64,7 +65,6 @@ class FakePeerConnection extends EventTarget { function fixture(options: { hangPreparation?: boolean; - expectedAudio?: boolean; } = {}) { let listener: ((event: NativeClientEvent) => void) | null = null; let peer: FakePeerConnection | null = null; @@ -109,7 +109,6 @@ function fixture(options: { "share_123456", control, onFailed, - options.expectedAudio, ); return { bridge, @@ -131,6 +130,70 @@ afterEach(() => { }); describe("native media bridge", () => { + it("releases an unstarted Browser reservation without sending an edge command", async () => { + const current = fixture(); + current.bridge.dispose(); + current.bridge.dispose(); + await expect(current.bridge.start()).rejects.toBeInstanceOf(NativeMediaBridgeError); + expect(current.peer().close).toHaveBeenCalledOnce(); + expect(current.control.prepareLocalEdge).not.toHaveBeenCalled(); + expect(current.control.closeEdge).not.toHaveBeenCalled(); + expect(current.onFailed).not.toHaveBeenCalled(); + }); + + it.each(["MediaStream", "RTCPeerConnection"])("preserves %s initialization failure without preparing an edge", (api) => { + const current = fixture(); + current.bridge.dispose(); + vi.clearAllMocks(); + const cause = new DOMException("Browser media initialization rejected", "NotAllowedError"); + vi.stubGlobal(api, class { constructor() { throw cause; } }); + let failure: unknown; + try { new NativeMediaBridge("share_123456", current.control, current.onFailed); } + catch (error) { failure = error; } + + expect(failure).toBeInstanceOf(NativeMediaBridgeError); + expect(failure).toBeInstanceOf(NativeMediaBridgeInitializationError); + expect((failure as Error).cause).toBe(cause); + expect(debugError).toHaveBeenCalledWith("native-bridge", "initialization-failed", cause); + expect(current.control.prepareLocalEdge).not.toHaveBeenCalled(); + expect(current.control.closeEdge).not.toHaveBeenCalled(); + expect(current.onFailed).not.toHaveBeenCalled(); + }); + + it("records the negotiation cause before retiring a failed bridge", async () => { + const current = fixture(); + const cause = new DOMException("Remote offer rejected", "OperationError"); + current.peer().setRemoteDescription.mockRejectedValueOnce(cause); + await expect(current.bridge.start()).rejects.toMatchObject({ cause }); + expect(debugError).toHaveBeenCalledWith("native-bridge", "failed", cause, + { type: "negotiation", state: "starting" }); + expect(current.peer().close).toHaveBeenCalledOnce(); + expect(current.control.closeEdge).toHaveBeenCalledOnce(); + expect(current.onFailed).not.toHaveBeenCalled(); + }); + + it.each(["candidate-signal", "candidate-apply"])("preserves the active %s failure and retires once", async (stage) => { + const current = fixture(); + const starting = current.bridge.start(); + await vi.waitFor(() => expect(current.peer().setLocalDescription).toHaveBeenCalledOnce()); + current.peer().emitTrack({ kind: "video", stop: vi.fn() } as unknown as MediaStreamTrack); + current.peer().setState("connected"); + await starting; + const cause = new DOMException("Connection is unavailable", "InvalidStateError"); + if (stage === "candidate-signal") { + vi.mocked(current.control.acceptSignal).mockRejectedValueOnce(cause); + current.peer().dispatchEvent(Object.assign(new Event("icecandidate"), { candidate: null })); + } else { + current.peer().addIceCandidate.mockRejectedValueOnce(cause); + current.emit({ version: 9, type: "edge-candidate", shareId: "share_123456", + connectionId: current.bridge.connectionId, candidate: null }); + } + await vi.waitFor(() => expect(current.onFailed).toHaveBeenCalledOnce()); + expect(debugError).toHaveBeenCalledWith("native-bridge", "failed", cause, { type: stage, state: "active" }); + expect(current.peer().close).toHaveBeenCalledOnce(); + expect(current.control.closeEdge).toHaveBeenCalledOnce(); + }); + it.each(["queued", "active"])("keeps media alive after a %s candidate is rejected", async (phase) => { const current = fixture(); const peer = current.peer(); @@ -224,8 +287,8 @@ describe("native media bridge", () => { }); it("does not publish a declared audio share before its audio track arrives", async () => { - const current = fixture({ expectedAudio: true }); - const starting = current.bridge.start(); + const current = fixture(); + const starting = current.bridge.start(true); let resolved = false; void starting.then(() => { resolved = true; @@ -347,6 +410,7 @@ describe("native media bridge", () => { expect(current.onFailed).not.toHaveBeenCalled(); expect(current.peer().close).toHaveBeenCalledOnce(); expect(current.control.closeEdge).toHaveBeenCalledOnce(); + expect(debugError).not.toHaveBeenCalled(); }); it.each(["prepare", "answer"])("does not signal after disposal during %s", async (stage) => { diff --git a/tests/native-viewer-peer.test.ts b/tests/native-viewer-peer.test.ts index b754997..320fe28 100644 --- a/tests/native-viewer-peer.test.ts +++ b/tests/native-viewer-peer.test.ts @@ -384,10 +384,16 @@ it("still disables native on a genuine failure after a signaling failure", async } }); -it.each(["route", "viewer"] as const)("keeps %s bridge failure unavailable across replacement peers", async (recoveryOwner) => { +it.each([ + ["route", "initialize"], ["route", "start"], + ["viewer", "initialize"], ["viewer", "start"], +] as const)("keeps %s bridge %s failure unavailable across replacement peers", async (recoveryOwner, failureStage) => { vi.stubGlobal("window", globalThis); vi.stubGlobal("MediaStream", class { getTracks() { return []; } }); - vi.stubGlobal("RTCPeerConnection", class { close() {} }); + vi.stubGlobal("RTCPeerConnection", class { + constructor() { if (failureStage === "initialize") throw new DOMException("Peer unavailable", "NotSupportedError"); } + close() {} + }); const browserSignal = vi.spyOn(ViewerPeer.prototype, "acceptSignal").mockResolvedValue(); const bridgeStart = vi.spyOn(NativeMediaBridge.prototype, "start").mockRejectedValue(new Error("bridge failed")); const receiveOffer = vi.fn(async () => ({ @@ -440,7 +446,7 @@ it.each(["route", "viewer"] as const)("keeps %s bridge failure unavailable acros await replacement.acceptSignal("parent", offer("replacement")); expect(browserSignal).toHaveBeenCalledTimes(2); expect(receiveOffer).toHaveBeenCalledOnce(); - expect(bridgeStart).toHaveBeenCalledOnce(); + expect(bridgeStart).toHaveBeenCalledTimes(failureStage === "initialize" ? 0 : 1); expect(acquire).toHaveBeenCalledTimes(3); } finally { replacement.dispose(); diff --git a/tests/route-transition.test.ts b/tests/route-transition.test.ts index c7dbe3c..48856d7 100644 --- a/tests/route-transition.test.ts +++ b/tests/route-transition.test.ts @@ -1109,23 +1109,11 @@ describe("minimal route transition contracts", () => { const log: string[] = []; const messages: ClientMessage[] = []; const streams: MediaStream[] = []; - const decodedSamples: Array<{ - framesDecodedDelta: number | null; - revision: number; - mediaIdentity: string; - connectionId: string; - }> = []; const subscribers: ReturnType[] = []; const route = new ViewerSfuRoute("viewer_12345678", { activatePeer: () => true, reconcileSfuChildren: () => undefined, onSfuStream: (stream) => streams.push(stream), - onSfuDecodedFrameSample: ( - framesDecodedDelta, - revision, - mediaIdentity, - connectionId, - ) => decodedSamples.push({ framesDecodedDelta, revision, mediaIdentity, connectionId }), send: (message) => { messages.push(message); return true; @@ -1151,15 +1139,6 @@ describe("minimal route transition contracts", () => { subscribers[0]?.events.onStream(firstStream); subscribers[0]?.events.onFirstDecodedFrame(); await vi.waitFor(() => expect(streams).toEqual([firstStream])); - subscribers[0]?.events.onDecodedFrameSample(3); - expect(decodedSamples).toEqual([ - { - framesDecodedDelta: 3, - revision: 1, - mediaIdentity: expect.stringContaining("publication_generation_12345678:"), - connectionId: sfuConfig(1).connectionId, - }, - ]); route.accept({ revision: 2, @@ -1173,9 +1152,7 @@ describe("minimal route transition contracts", () => { subscribers[1]?.events.onStats?.({ intervalFramesDecoded: 2, } as ConnectionMetrics); - subscribers[1]?.events.onDecodedFrameSample(null); expect(messages.filter((message) => message.type === "route-ready")).toEqual([]); - expect(decodedSamples).toHaveLength(1); subscribers[1]?.events.onFirstDecodedFrame(); subscribers[1]?.events.onFirstDecodedFrame(); @@ -1190,17 +1167,6 @@ describe("minimal route transition contracts", () => { assignment: viewerSfuAssignment([], "publication-generation-2"), }); await vi.waitFor(() => expect(streams).toEqual([firstStream, pendingStream])); - subscribers[1]?.events.onDecodedFrameSample(null); - subscribers[0]?.events.onDecodedFrameSample(9); - expect(decodedSamples).toHaveLength(2); - expect(decodedSamples[1]).toMatchObject({ - framesDecodedDelta: null, - revision: 2, - connectionId: sfuConfig(2).connectionId, - }); - expect(decodedSamples[1]!.mediaIdentity).not.toBe( - decodedSamples[0]!.mediaIdentity, - ); expect(subscribers[0]?.deactivate).toHaveBeenCalledOnce(); expect(subscribers[0]?.disconnect).toHaveBeenCalledOnce(); }); diff --git a/tests/viewer-route-health.test.ts b/tests/viewer-route-health.test.ts new file mode 100644 index 0000000..b41dd0b --- /dev/null +++ b/tests/viewer-route-health.test.ts @@ -0,0 +1,67 @@ +import { readFileSync } from "node:fs"; +import { createContext, runInContext } from "node:vm"; +import ts from "typescript"; +import { afterEach, expect, it, vi } from "vitest"; +import { EMPTY_METRICS } from "../src/client/types"; +import type { ViewerPeerEvents } from "../src/client/webrtc/viewer-peer"; + +// Exercise the page's actual media callbacks, without needing a physical game. +const source = ts.createSourceFile("ViewerPage.tsx", readFileSync( + new URL("../src/client/pages/ViewerPage.tsx", import.meta.url), "utf8", +), ts.ScriptTarget.Latest, true, ts.ScriptKind.TSX); +const owners = new Set(["ensurePeer", "reportActivePeerFailure"]); +const functions: string[] = []; +function collect(node: ts.Node): void { + if (ts.isFunctionDeclaration(node) && node.name && owners.delete(node.name.text)) { + functions.push(node.getText(source)); + } + ts.forEachChild(node, collect); +} +collect(source); +if (owners.size) throw new Error("Viewer route owner was not found"); +const executable = ts.transpile(functions.join("\n"), { target: ts.ScriptTarget.ES2022 }); + +afterEach(() => vi.useRealTimers()); + +it("retains a connected route through quiet video and still reports actual recovery exhaustion", () => { + vi.useFakeTimers(); + let events!: ViewerPeerEvents; + const peer = { + isRecovering: () => false, + getConnectionIdentity: () => ({ parentPeerId: "host", connectionId: "edge" }), + hasConnectionId: (id: string) => id === "edge", + }; + const send = vi.fn(); + const reportPeerFailure = vi.fn(() => true); + const setPeerSnapshot = vi.fn(); + const context = createContext({ + Date, active: true, currentHostOnline: true, + currentIceConfig: { iceServers: [] }, currentRoutePolicy: { natPrediction: false }, + currentRouteRevision: 1, currentRouteAssignment: { upstream: { kind: "peer", peerId: "host" } }, + activePeerMetrics: null, peerRef: { current: null }, + signal: { send }, viewerSfuRoute: { reportPeerFailure }, + createViewerMediaPeer: (_config: unknown, callbacks: ViewerPeerEvents) => { events = callbacks; return peer; }, + createOwnedViewerRestartSender: () => vi.fn(), + offerPeerQualityEvidence: vi.fn(), setPeerSnapshot, + dispatchPresentation: vi.fn(), connectionFact: (state: string) => state, + }); + runInContext(executable + "\nensurePeer();", context); + const snapshot = { + peerId: "host", connectionId: "edge", connectionState: "connected" as const, + metrics: { ...EMPTY_METRICS, intervalFramesDecoded: 0 }, + }; + // Silence also occurs for a live, minimized source; elapsed time cannot locate + // the fault in a route, even if audio or incomplete video packets still arrive. + for (let second = 0; second < 40; second += 2) { + vi.advanceTimersByTime(2_000); + events.onUpdate(snapshot as Parameters[0]); + } + events.onUpdate({ ...snapshot, metrics: { ...snapshot.metrics, intervalFramesDecoded: 30 } } as Parameters[0]); + expect(send).not.toHaveBeenCalled(); + expect(reportPeerFailure).not.toHaveBeenCalled(); + expect(setPeerSnapshot).toHaveBeenLastCalledWith(expect.objectContaining({ + metrics: expect.objectContaining({ intervalFramesDecoded: 30 }), + })); + events.onRecoveryExhausted?.("host", "edge"); + expect(reportPeerFailure).toHaveBeenCalledExactlyOnceWith("host", "edge"); +});