Piik/docs/research/node-local-adaptation.md

Node-Local Adaptive Encoded Reuse

Reviewed: 2026-09-07. ADR-0013 now owns the accepted model and highest-demand envelope. Codec backends and timing values still need implementation evidence. ADR-0007 remains the runtime contract. The design proposal narrows implementation from the measurements below; it does not reopen the agreed topology/reuse model.

Agreed Model

ADR-0013 is the sole owner of the accepted direct-child model and policy. This file retains evidence and alternative comparisons, not a second design contract. The implementation design is owned by node-local media design. base/native names the selected source quality; it is not a second output alongside an identical high encoding.

Two Workstreams

• Self-contained Piik Server: library-level STUN/media components inside the executable, not bundled external service processes or an automatic installer.
• Adaptive reuse: local derivation, shared lower outputs, independent child delivery and quality recovery, usable by Native App and the server.

These share media-source and lifetime concepts but need separate acceptance. A forwarding-only SFU library does not provide the second workstream by itself; server-side derivation needs real decoding/encoding capacity. Pure Browser peers can consume normal WebRTC outputs and benefit from capable upstreams, but have no generic cross-PeerConnection encoded-forwarding API.

Existing Building Blocks And Gaps

• mediaedge.Source already forwards encoded RTP to multiple Pion edges.
• Native GCC observations currently drive route evidence, not the shared encoder. A sustained weak edge may use a measured Browser-sender exception. That exception is not yet a reusable derived-encoding pool.
• Existing capture encoders, generation fences, keyframe requests and bounded route operations are reusable. A new standalone quality controller, global node score or periodic rebalancer is not assumed.
• Current routing reacts primarily to persistently degraded senders. A healthy edge forwarding already-low input may not trigger a move. Transport health and available source quality must be distinguished before claiming recovery.
• Simulcast, encoder allocation, active encoding, layer forwarding and bandwidth probing are different operations. Pausing a representation must not be mistaken for destroying its codec object or reopening capture.

Open Choices

• Always encode all configured lower outputs, enable all after the first weak child, or activate only the representations currently demanded.
• Keep unused encoder resources warm or release them; decide from cold-start, first-keyframe, GPU-session, memory and idle-CPU measurements.
• Use a bounded representation set and one parameter owner per shared output. Determine which mature component owns demand grouping and adaptation; do not assume that independent child GCC loops can all mutate one shared encoder.
• Coordinate a usable-frame handoff before switching, preserve unaffected branches, and avoid repeated downgrade/recovery churn.
• Check server software-transcode cost and platform-native hardware availability independently. A network-capable parent is not necessarily a capable encoder.

Simulcast And Dynacast Findings

These observations were checked against LiveKit Server 1.13.6 and JS SDK 2.22.1, not inferred from the words "automatic" or "simulcast":

• Simulcast advertises several encoded representations of one source. The SDK constructs a finite encoding set before publishing; it does not create an arbitrary new representation for every bandwidth sample or subscriber.
• The pinned screen-share defaults create the original plus one lower spatial representation for ordinary screen sizes, not invariably three encoders.
• The server's Dynacast video manager aggregates the highest subscribed quality across subscribers/downstream SFU nodes. Its update enables existing qualities at or below that maximum. Thus a high requirement can retain lower fallback representations; this is not exact per-current-layer consumer reference counting. Do not claim every unused individual lower layer is always stopped.
• A higher quality request is not subject to downgrade-only debounce. Reduced maximum demand is debounced; the pinned server default is five seconds. This is an upstream implementation fact, not a proposed Piik timing constant. Publisher deactivation debounce is not a five-second delay before reducing traffic to a congested subscriber.
• The SDK applies the selected active flags through RTCRtpSender.setParameters on the existing sender. WebRTC's active=false preserves SSRC identity. It is not a guarantee that every Browser releases, or retains, hardware encoder resources in the same way.
• Subscriber forwarding choice is separate from publishing activity and from destroying an encoder. A decoder-safe layer switch needs an appropriate random-access point and consistent RTP/timestamps, not just another byte array.
• LiveKit's prober documents padding-based capacity probes independently of video-layer production. Higher-video-layer probing is another possible strategy, but cannot be substituted for a calibrated congestion controller.

The useful lessons are finite representations, distinct demand/transport ownership, asymmetric activation/deactivation and independent network probing. They do not imply importing the whole LiveKit service or extending its subscriber-tree demand aggregation to override our direct-child derivation rule.

Strategy Comparison

Shared encoding and per-child selective delivery are common to both compared policies. The owner explicitly excludes Native-plus-Browser and repeated full encoders as the next experiment's reference baseline. Compare exact demanded outputs against the maximum-demand envelope, not shared versus unshared media.

Strategy	Main benefit	Cost to verify
Base plus all lower encodings always active	Lower outputs are already being produced when needed	Permanent encode/GPU cost even for an entirely healthy tree; still needs a switch point
Base only, then all lower outputs after any weak child	Small policy, some idle savings	First downgrade remains a cold start; one weak child starts outputs nobody requested
Finite outputs activated only as needed	Avoids unused representations and shares compatible demands	Cold start and churn if codecs are constantly destroyed/recreated
Demand activation with bounded warm retention or a minimal fallback output	Potential compromise between steady cost and transition delay	Requires measurements to justify retention or pre-encoding; not free memory/session capacity

The owner's 2026-09-07 preference makes the LiveKit-style maximum-demand envelope a first-class candidate: keep the highest requested representation and its lower fallbacks active; stop unused higher outputs when all direct children need less. Compare it with exact demand activation, not with an assumption that exact demand is inherently better. Prefer the simpler envelope if its measured cost is small and switching is better. Good network capacity alone does not prove spare encoding capacity or cheap lower outputs.

Both policies retain the original source authority and the ability to recover; stopping an unneeded high output does not lower capture or discard the best received input. Exact demand can retain inactive codec resources for a bounded comparison instead of repeatedly destroying them. No production timing value, extra polling loop or prediction score has been selected. Do not blindly create a fallback encoder set at every forwarding relay: that would add decode/encode work to the entirely healthy tree. Reusing received encodings and deciding how an already-needed encoder set stays active are separate decisions; publisher measurements cannot stand in for relay costs. Where the media framework supports it, negotiate the finite representation capability up front and activate existing identities; a quality change should not reopen capture or recreate a PeerConnection merely to add an encoder.

Preparing/retaining an encoder, actively encoding frames and forwarding packets are separate costs. Share stop, source replacement and failed ownership still retire their resources immediately; warm retention cannot revive a retired source.

Reusable Components

Component	Useful capability	Boundary
Existing mediaedge.Source and Pion RTP/WebRTC/interceptors	Encoded fanout, per-edge transport and feedback	No decoder/transcoder or shared-variant policy is supplied automatically
Existing Native encoders and generation/keyframe ownership	Platform encode paths already integrated with Piik	Received-source decode/scale and shared derived outputs still need design
LiveKit Dynacast, layer selector and prober sources	Reference for demand aggregation, safe switching and recovery probes	Apache-2.0 code; inspect imports/ownership before any extraction, not whole-service embedding
libvpx multi-resolution VP8 API/example	Multiple spatial encodes through one coordinated API; libvpx is already used	VP8-specific; no promise that H264 hardware sessions have the same cost curve
OpenH264 AVC simulcast API	Library H264 encode/decode and per-spatial-layer configuration	Software codec; inspect realtime cost and actual source/binary distribution terms before selection
FFmpeg libavcodec and frame/filter APIs	Embedded decoding, scaling and hardware/software encoding	A library option, not launching ffmpeg; build/license/size cost remains to compare
Pion mediadevices	Go codec-controller and raw-frame broadcast interface reference	Cgo/native codecs still exist; not a reason to replace the working capture adapters
GStreamer tee/queues and codec elements	Shared decode/frame flow and isolated output branches	webrtcsink currently encodes per consumer, so it does not directly solve shared derived output

No candidate has yet demonstrated the complete local derivation, reuse and recovery contract. The current inLive probe proves forwarding only. A small SFU library does not by itself replace a transcoder. Do not add FFmpeg/GStreamer or change the current Native codec stack on documentation claims alone.

Focused Comparison Plan

Use the same source, codec, frame rate and direct-child count for each arm:

1. Healthy children: verify no derived encode is added just for fanout.
2. One weak child, then two compatible weak children: verify one derived output is sufficient for both and the original branch stays unchanged.
3. Bandwidth drop/recovery and short oscillation: compare cold versus retained encoder startup, first usable frame, freezes, CPU/GPU, memory and actual RTP traffic. Count extra decoders and CPU/GPU frame copies as well as encoders; Browser-decoded preview frames are not automatically zero-copy Native input. Measure extra IDR/probe traffic, not only the selected bitrate.
4. Low inherited input on an otherwise healthy edge: distinguish lack of better source from lack of link capacity; exercise higher-input recovery or the existing bounded route operation without a new global optimization loop.
5. Source replacement, child departure and stop: verify generations and release of decoder/encoder surfaces, tracks and queues.

The first CPU-only encoder comparison is complete. It compares existing Native VP8 encodes and scripted direct-child demand, not network adaptation or playback. It supports keeping the maximum-demand envelope as a candidate, but also exposes timestamp-sensitive resume behavior in a naive warm-encoder composition. Validate mature multi-resolution pause/resume semantics before implementing retention. H264 hardware, relay decode/scale and actual receiver switching remain open; no production policy changed. The subsequent software-preference decode comparison now includes VP8/H264, scaling and two real VideoDecoders, plus received-source relay derivation. It measures local component readiness only, not WebRTC delivery or congestion. The design treats representation bookkeeping as small and puts the remaining overhead work in codec lifecycle, shared decode/surfaces and the transport library's switching/probing path.

Primary References

• LiveKit JS encoding construction, 2.22.1
• LiveKit JS publishing layer activation, 2.22.1
• LiveKit Dynacast quality aggregation, 1.13.6
• LiveKit activation/debounce, 1.13.6
• LiveKit default pause delay, 1.13.6
• LiveKit capacity probing design, 1.13.6
• WebRTC active encoding semantics
• libvpx multi-resolution example
• libvpx multi-encoder API
• OpenH264 encoder API
• OpenH264 AVC simulcast release history
• FFmpeg codec send/receive API
• Pion mediadevices codec boundary
• GStreamer branch isolation
• GStreamer WebRTC consumer encoding

中文确认

核心是“只负责直属下游”：有合适编码就转发，缺少低档才由直属上级按需派生， 相同需求复用派生输出。不是 Host 为全树预编码所有档位，也不是每条弱边独立 编码。Host 同样遵守此规则。基准档、常用档的名字不能造成重复编码。

拓扑优化负责修复上游输入质量受限，局部媒体模块负责可用表示的派生、复用和 选择。稳定传输低画质不等于路径已经最优；现有优化不能直接视为已覆盖这种 恢复。当前先研究按需/预编码策略及成熟实现，尚未改动生产媒体或路由。

优先把 LiveKit 式“最高需求档及以下保持编码”与精确按需启用作为平等候选， 不预设后者更优；若常备低档成本小、切换更好，就选择更简单的策略。所有直属 下游都不需要高档时可以停掉高档输出，但不修改捕获或丢弃最好的输入来源。 先用已有 VP8 编码器短时测量成本和关键帧启停，不把结果外推成 H264 硬件或 真实网络验收。具体库、档位与启停时间仍未作为生产决策。