## Problem and outcome / 问题与结果 This phase adds lightweight room interaction without making media depend on chat, and adds Windows source-audio exclusion and optional SFU-only hosting at their existing owners. It also corrects confirmed encoder/capture ownership and native color-conversion defects. Host microphone commentary remains on the shared media path; room voice and 4K are outside this phase. ## Release notes ### 中文 这是一次功能更新:Piik 加入了文字聊天与表情互动,并完善了 Windows 声音采集、画面颜色和自部署选项。 #### 新功能 - **边看边聊。** 在可移动、可缩放的聊天窗口中发送消息,或开启弹幕。停止分享后,房间中的朋友仍可继续聊天。 - **表情互动。** 点击沙发上的人物,给自己发表情,或把番茄等表情送给朋友;各端按同一事件显示动画。 - **排除应用声音。** Windows App 分享整个屏幕时,可排除一个应用及其子进程的声音,适合同时使用语音软件。所选应用退出后,来源声音停止,避免恢复为未过滤的系统声音。 - **可选的纯 SFU 模式。** 自部署管理员可让共享画面和声音统一通过媒体服务器转发。默认仍为优先 P2P;App 公网邀请保持纯 P2P。 - **Windows HDR 转 SDR。** 原生采集与缩略图自动将 HDR 来源转换为 SDR,沿用现有观看链路。 #### 修复与改进 - 修复原生颜色转换的范围与矩阵处理,避免已复现的暗部丢失、高光裁切和颜色偏差。 - 修复共享编码器切换时使用过期需求或过期帧信息的问题,保护正在观看的画面;修复换源后旧采集事件可能影响新来源的问题。 - 统一新控件、浮窗与来源选择文案,补充中英文使用说明。 #### 使用与升级 聊天与表情需要新版站点页面和 Server;声音排除需要新版页面与支持此能力的 Windows App。聊天只保留本页最近收到的消息,刷新后清空,不补发历史,且不提供端到端加密。 HDR 转换输出为 SDR,不代表端到端 HDR 或多声道支持。既有房主麦克风与立体声能力保留。 ### English This feature update brings text chat and reactions to Piik, alongside improvements to Windows audio capture, picture colors and self-hosting options. #### New features - **Chat while watching.** Send messages in a movable, resizable window, or enable on-screen chat. Friends can keep chatting after the host stops sharing. - **Reactions.** Select a person on the sofa to react above yourself or send a tomato and other reactions to a friend. Participants animate the same shared event. - **Exclude application audio.** When sharing a whole screen with Windows App, exclude one application and its child processes, useful when using a separate voice app. Source audio stops if the selected app exits, preventing a return to unfiltered system sound. - **Optional SFU-only hosting.** Site operators can require shared video and audio to pass through the media server. P2P-first remains the default; App Public invite remains P2P-only. - **Windows HDR-to-SDR conversion.** Native capture and source thumbnails automatically convert HDR sources to SDR for the existing viewing path. #### Fixes and improvements - Correct native color-conversion ranges and matrices, addressing reproduced crushed shadows, clipped highlights and color shifts. - Correct stale demand and frame evidence during shared-encoder handoffs to protect the current picture. Prevent events from a replaced capture from affecting its successor. - Align new controls, floating panels and source-picker copy, with updated Chinese and English guides. #### Using this update Chat and reactions require updated site pages and Server. Audio exclusion requires updated pages and a Windows App that supports the capability. Chat retains only recent messages received by the current page, clears on refresh and does not replay history. It is not end-to-end encrypted. HDR conversion produces SDR; it does not add end-to-end HDR or surround sound. Existing host microphone commentary and stereo support remain available. ## Verification / 验证 - Web type checks, 78 test files / 1275 tests and production build: PASS. - Go core tests, formatting/vet and target builds: PASS; Darwin cgo remains platform-limited on this Windows runner. - Native build/check/color probes and synthetic HDR-to-SDR H.264/VP8 browser reception: PASS. - Access/privacy gate, Native H.264 SFU-only media gate and local Auto/H.264 Host gate: PASS. - Three real production pages: chat/reactions, stop/resume, idle-room join, host departure/return, offline draft recovery and invitation revocation: PASS. - Synthetic parent/child process-audio exclusion and target lifetime: PASS; no raw audio retained. - Windows candidate archive extraction and packaged Local startup/shutdown: PASS. - Documentation and website build: PASS. - [Candidate CI](https://github.com/TNTcraftHIM/Piik/actions/runs/36873766646) at `f26cd208d03bc0dcbe353f8cac217f993216b0a3`: Go core and race checks, Web tests, Server/container checks and Windows/macOS/Linux App packaging all PASS. Publication was skipped as intended. The owner accepted the UI/model and bounded local walkthrough for phase closure. Physical HDR/mixed displays, mobile keyboard, real voice applications across output devices and macOS/Linux capture remain unverified. Existing field reports without matched evidence are not claimed resolved. The owner authorized v1.7.0 publication on 2026-10-02. The final documentation-only commit records that authorization; product code matches the verified candidate above. ## Impact / 影响 Authorized version: **v1.7.0**, matching the local release planner. Current wire contracts remain Browser/server 23, Native control 9 and capture 7. Optional capabilities gate new functionality; existing room/media authority remains owned by the existing services. No new ports, message persistence, room voice or automatic room creation on page visit. Updated README, guides, standards, research evidence and TODO stay with the candidate phase.
17 KiB
Presentation And Lifecycle
This file owns user-visible workflow, presentation authority, presence, and Browser lifecycle promises. Component structure, CSS, exact copy, and routine UI tests belong in source code.
Product Surface
The first screen is the working tool, not a marketing page. Once site access is
available, starting a share and joining a room are equally discoverable actions;
joining expands the four-digit room form in place, while /join remains a direct
entry route. Users never choose a media topology.
Host entry requires both site access and a validated capability description. Initial entry, retry and password recovery load the same prerequisites; failed authentication cannot leave a later successful entry using absent capability state. A capability-load failure remains retryable and does not imply that a service is available or disabled. Viewer/Join entry does not need this Host-only description.
The Host workspace centers the capture preview, room code, invitation and common share controls. The Viewer workspace centers one persistent 16:9 video stage, room identity, current state, roster, and one manual media-reconnect action. Controls and text must remain usable without overlap or horizontal overflow on desktop and mobile viewports.
A capable, opted-in authenticated room session outlives its current publication. Stopping capture retires media, not chat; restoring an existing room joins it without starting capture. Entering the page alone does not create a room. Room close, lost authority and page teardown still retire the session. Connected Viewers may keep interacting while the Host page is absent, provided the room service and their admitted sessions remain available. Rooms and access owns interaction authority and retention; status owns release readiness.
Piik App opens this same application in the system Browser. Its small startup surface selects Local, temporary public invitation, or a configured Site before entering the Host workspace. Viewer links opened at that activated origin may use the same running App without changing the Viewer UI. Starting a share offers the Browser picker and any exact App-owned native windows. It never chooses a window automatically. A reproduced Browser-window, capture, or background failure is required before introducing an embedded Web runtime.
Opening the system Browser is an asynchronous convenience action. One pending OS handoff is allowed; a slow or failed handler cannot block readiness or App exit. The console retains the entry address and reports the underlying error. The App does not terminate the user's browser to enforce a handoff deadline.
Server and App share one control layout and vocabulary. Deployment capabilities may fix a control on or off, but do not remove its place in the interface; disabled controls explain their reason in text and pure-visual modes. Real workflow differences remain explicit: site authorization, opt-in diagnostics and the capture/audio capabilities of the connected device are not alternate versions of the product UI.
A remembered App activation permits discovery when choosing a source or receiving media; it does not make App a prerequisite for sharing. Idle Host pages hold no App control session. The shared source selector orders Browser, Camera, Window and Screen, defaulting to Window after App detection. Without an App it shows only Browser and Camera, defaulting to Browser. Discovery never blocks Browser selection or overrides an explicitly chosen tab. Refresh remains reachable after failed App discovery; pending attempts are shared, completed failures do not suppress later discovery. Cancelling the selector or ending a share releases its unused control session; an obsolete operation cannot install or retire another operation's media. Camera thumbnails own only temporary preview inputs; leaving their tab or picker retires them. An active shared camera is borrowed, never reopened or stopped by preview. Optional native ingress uses an already available connection without another scan. If it is not ready at Browser startup, that share keeps Browser senders.
Native Host startup acquires its required Browser media connection before starting capture. Cancellation owns that connection immediately, while native share ownership and required audio tracks come from the accepted App response. Browser initialization failure remains distinct from capture permission failure.
A reachable Piik App with an incompatible control protocol is distinct from an absent App. Discovery continues looking for a compatible process before reporting the mismatch; the source selector retains Browser operation and offers update/ refresh guidance. Discovery metadata may grow, while known identity and capability types remain validated. Missing optional capabilities are unavailable; control commands and responses retain their strict protocol contract. An empty source list means enumeration succeeded for the selected category. Connection, capture-capability and enumeration failures keep their own feedback. Discovery has one bounded deadline that allows time for browser local-network consent; a failed fetch alone cannot establish that consent was denied. Optional Viewer reception uses Browser media immediately when the browser reports that local App access still needs consent or is denied. It rechecks permission on a later connection attempt; a later grant does not replace healthy Browser media. App Local needs no cross-address-space consent. Browsers without a permission query retain bounded discovery.
Visual Language
Chinese, English, and pure-visual modes are three expressions of the same typed
state and command catalog. On first use, the App and Server UI match the
Browser's primary system language against registered locales, using English
when none matches. Current catalogs cover Simplified Chinese and English.
Both Simplified and Traditional Chinese system locales select the current
Chinese catalog across App, Server UI and website; this includes region and
script tags such as zh-CN, zh-TW, zh-HK, zh-Hans and zh-Hant.
Pure-visual mode remains an explicit choice. An explicit later choice is
persisted and takes precedence over that default, including App launch handoff.
Explicit launcher choices apply only to the presentation domains the user chose.
Older launchers cannot distinguish defaults from explicit choices; updated Sites
preserve their saved preferences when receiving those unmarked hints.
The App console uses its own supported presentation languages and falls back
to English for additional UI languages. Text modes use concise copy.
The visual language is the single owner of illustrative roles and objects, semantic colour, panel grammar, motion, accessible input and responsive hierarchy.
Playback Ownership
One <video> element owns Viewer playback and frame proof. The shared playback
bar reflects that element and provides local play/pause, mute, volume, theater,
picture-in-picture, fullscreen and the existing reconnect action. It stays reachable while waiting
or disconnected. Theater and fullscreen are separate modes. Page-element
fullscreen includes the playback bar; Safari's video-only fullscreen uses
system controls and native video events to reflect playback and exit.
Identity, connection details and topology stay in the deck.
Replacement video bindings do not consult Viewer-local pause: when the Host is
playing, they attempt playback subject to browser autoplay permission. There is
no separate remembered playback intent for source changes or reconnects. A new
binding received while the Host is paused waits for the Host to resume; it does
not inherit the previous binding's Viewer-local pause.
Adding, replacing or removing audio on the same video binding updates only its
audio consumer; it preserves current-frame proof and Viewer-local playback.
A pending play() canceled by pause or rebinding is not a media failure;
cancellation alone must not invalidate frame proof or disable local playback.
The bar spans the screen's lower edge, preferring one row. Narrow screens shorten the volume slider and compact horizontal spacing and button widths, retaining 44px control height and all actions; the percentage readout hides on narrow screens. If the control groups cannot fit, they wrap without overlapping. Playing video hides the bar after two idle seconds, following Media Chrome's default. Pause or unavailable playback keeps it visible. Control hover, keyboard focus, dragging and an open tooltip hold it open; mouse movement reveals it, and a touch tap on the picture toggles only visibility. One presentation-only idle timer owns visibility; it never changes playback or route state. Controls follow the shared motion grammar.
A desktop primary click on the picture toggles local playback after a brief double-click window; double-click toggles the available fullscreen action without first toggling playback. Controls, overlays and keyboard actions keep their own immediate behavior.
Volume starts on native audio output. An explicit setting away from 100% activates
a local Web Audio stream source and gain, from silence to 200% amplitude. This also
avoids iPhone's system-owned video.volume. This output
exclusively owns sound while running, follows video pause and user mute, and
rebinds when the audio track changes. It never modifies the received track or
the stream relayed downstream. Unmount closes the graph. Browsers without gain
support retain 0–100%; video-only system fullscreen also hands audio back to the
native controls at at most 100%, rather than running two outputs.
This uses the standard Web Audio stream-source and gain nodes. The gain is amplitude scaling, so loud source peaks can clip above 100%.
Picture-in-picture uses the browser's native video window, including Safari's presentation-mode API. Native enter/leave events own its displayed state; closing the window or replacing the received stream does not create another playback owner. Successful entry exits this video's other fullscreen mode, and unmount closes only this video's window. Unavailable capabilities remain disabled with an explanation. The native window contains video and system controls, not page overlays or the custom bar. See the PiP specification and Safari integration.
Local playback actions never change Host capture or another Viewer. Reconnect retains the existing route-operation owner. The Host preview remains muted and has no media controls; explicit Host actions own authoritative pause and stop.
Autoplay rejection exposes the play action. A connection, track object,
or playing event alone is not proof that the current route is visible. The
Viewer removes its blocking presentation only after a current-generation frame
is composited, using requestVideoFrameCallback() where available and a decoded-
progress fallback otherwise.
State And Recovery
Access, Host presence and pause, route prepare/active, P2P/SFU connection, media binding, frame proof, autoplay, and terminal failure are typed facts interpreted by one presentation reducer. Older route or media generations cannot overwrite the current result, and raw server errors are never rendered directly.
The room route revision orders control messages; it is not a Viewer media identity. Reauthentication replaces the route snapshot authoritatively, even if a replacement controller restarted revision numbering; ordinary updates remain monotonic within that authority. A Viewer frame belongs to its local media generation and remains current across unrelated graph revisions until that exact binding is replaced, invalidated, or terminally failed. Pending candidates never own the primary badge, status line, or overlay while committed media remains proved. A shared visual projection maps those facts to title, media status, notices and overlays. None of these views owns state for another. Connection availability, actual playback and quality limitation remain separate; raw transport snapshots do not prove playback or quality. The status preview records the visual vocabulary and the evidence needed for participant lamps.
Excluding time spent in the Browser's capture/play authorization UI, a Viewer
request targets a first visible frame within three seconds. Until then, the page
must continuously show a truthful accessible connection stage rather than a
black screen, ICE-connected state, or unproved playing event.
- A proved current frame remains visible behind non-terminal recovery state.
- The status projection owns the television, Header, notices, participant lamps and title. These consumers cannot turn an unproved frame into playback or ordinary quality into failure.
- The Host's couch and Viewer overview share the
participant projection. The
Host's own pawn has no additional lamp. Committed
mediaReadyowns readiness; transport diagnostics and quality samples cannot override it in either direction. - Viewers do not infer another Viewer's quality or diagnose missing readiness as terminal failure; roster readiness and quality evidence remain distinct.
- An overlay explains the absence of a usable picture, authoritative Host pause, or a required user action. Ordinary quality warnings never cover playback.
- Host pause retains the current frame and waits for resume.
- Host signaling loss alone does not invalidate media that is still healthy.
- Exact media failure or terminal route failure invalidates current-frame proof before showing a retained frozen background and recovery or failure state.
- Room not found is distinct from current-room access denial and does not prompt for a password.
- Manual reconnect recovers the same route. It does not select a new parent.
- An actionable overlay may appear while fullscreen remains active; application state never forces the user out of fullscreen.
Decoded-frame silence is not a route-failure signal. A live source can stop repainting; media-framework failures, not an application silence timer, own recovery. First-frame admission and candidate-relative proof still require decoded progress. Visibility, page freeze, and pagehide invalidate quality observations, not already-proved playback. Returning to the page re-arms frame observation; an existing recovery state still requires a fresh current frame to clear. SFU media is retained independently of transient room-signaling loss, but Browser or OS suspension and page reclamation remain outside Web guarantees.
Failure of the local Native media bridge disables Native reception for the current Viewer session before route recovery runs. Future peers in that session use Browser reception; existing healthy peers are not torn down by this choice. Parent-network failure alone is not evidence that the local Native capability is unavailable.
Presence And Diagnostics
Host and Viewer pages show the authoritative online Viewer count and roster, independent of the media graph. A display name is Browser-local, NFC-normalized, at most 24 Unicode code points, and rejects unsafe control or bidirectional characters. It is a label, not an account, authorization identity, or routing input. Duplicate names are allowed and receive a room-scoped peer suffix only when disambiguation is needed.
Connection details are progressive and use one card shape for P2P, peer relay,
and SFU Viewers. Summary fields prioritize actual resolution, FPS, bitrate, and
loss; deeper fields appear only when meaningful. Topology is a separate view.
Native bandwidth or CPU limitation warnings describe the exact outbound sender
that reported adaptation; they do not by themselves locate the physical
bottleneck or describe the Viewer's receive path.
The Host's participant freeze warning requires fresh receive evidence matching
the current upstream and a positive freeze count or duration for that window.
Window *Delta fields never inherit older values; other fields may retain their
existing details-display cache. The existing evidence expiry removes stale
warnings. No additional participant state, broadcast, timer or quality score is
introduced. Detailed per-edge observations remain available without a global
warning selected from the first limited child.
Locally exposed selected-candidate addresses may be shown only on the Browser
that owns that PeerConnection and are never uploaded, persisted, or used for
identity or route selection. Explicit Debug mode can export a bounded diagnostic
report through the separate diagnostic owner;
ordinary connection details do not collect or expose that report.
Current scope includes Web Host, Web Viewer, Browser relay, and optional Native Host/Viewer media beneath those same pages. Desktop and mobile Browsers are Viewer targets; mobile Web capture, reliable background relay, App-owned SFU, and physical non-Windows native acceptance remain later platform work.