okooooooo/Piik
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(app): improve capture controls and sharing feedback

Browse Source 
## Problem and outcome / 问题与结果

Windows App users can now choose whether to show the capture border. The same border owner handles source previews and active capture; unsupported or denied requests keep normal capture available. The source picker distinguishes an absent/incompatible App, unavailable capture capability, failed enumeration and a genuinely empty list. Chrome's pending local-access permission no longer holds optional Viewer reception.

This phase also reduces tooltip obstruction and contains a failed Linux encoder output within its own branch. Browser/server v23, Native control v9 and capture v7 remain compatible. Closes #401.

## Release notes

### 中文

这是一次分享体验更新，重点改进 Windows 采集边框、App 选源反馈和提示浮窗，并修复 Linux 采集输出的故障隔离。

- **采集边框可选。** Windows App 选源器新增「显示采集边框」，位于声音开关左侧，默认关闭。支持的系统上，来源预览也会隐藏边框。
- **找不到窗口时，提示更明确。** 区分 App 未连接、版本不兼容、采集不可用和读取失败；授权后可以刷新列表重试。改善 Chrome 首次访问时的观看连接，避免本机授权等待拖慢入座。
- **提示浮窗更顺手。** 统一 500 毫秒悬停延迟，减少对画面和控件的遮挡，改善触屏长按和关闭操作；只用于说明的控件使用普通鼠标光标。
- **Linux 分享更稳。** 修复单路编码输出失败可能连带中断其他输出的问题。
- **排查文档更完整。** 补充 App 来源列表、Windows 10 黄框及浏览器 WebRTC 设置的排查说明。

使用新的采集边框开关，请更新 Windows App 并刷新已打开的网页。Windows 10 没有对应的系统接口，边框仍由系统显示；文档中心提供可选处理方法。系统权限或其他采集会话也可能要求保留边框。

### English

This sharing update improves Windows capture borders, App source selection and tooltips, and fixes failure isolation between Linux capture outputs.

- **Choose whether to show the capture border.** The Windows App source picker adds a **Show capture border** switch to the left of audio, off by default. Source previews also hide the border on supported systems.
- **Clearer source-selection feedback.** The picker distinguishes an unavailable or incompatible App, unsupported capture and a failed source list. Refresh after granting access to retry. Pending local-access permission in Chrome no longer delays Browser reception for Viewers.
- **Less intrusive tooltips.** A consistent 500 ms hover delay, improved placement and touch-hold behavior make guidance easier to read and dismiss. Help-only controls use the ordinary cursor.
- **More resilient Linux sharing.** A failed encoder output no longer interrupts healthy sibling outputs.
- **More useful troubleshooting.** The documentation covers App source lists, Windows 10 capture borders and browser WebRTC settings.

Update the Windows App and refresh open pages to use the capture-border switch. Windows 10 lacks the corresponding system API and retains its system border; an optional procedure is available in the documentation center. System permission or another capture session may also require the border to remain visible.

## Verification / 验证

- `npm run check`: TypeScript, 998 tests and production Web build passed.
- `npm run check:go`: Go formatting/tests/vet/target builds and Windows native capture checks passed. macOS cgo builds require their native CI runner.
- Documentation, repository hygiene and website build passed.
- Real Chrome 152 profiles: first-use local permission, recovery after consent and source-list refresh; optional Viewer discovery before/after consent; App Local without cross-address-space consent.
- Public invitation gate: link creation, remote page and WebSocket access, App exit and link retirement passed.
- Windows 11 border on/off, source preview and concurrent-capture checks have physical evidence. Tooltip hover, keyboard, touch-hold, dismissal and narrow-layout checks passed.
- [Candidate CI](https://github.com/TNTcraftHIM/Piik/actions/runs/35157993032) passed at `1d159831`: Server/container, Windows/Linux/macOS packages, Go race checks and frontend validation. Publication was skipped.
- Local Windows ZIP passed packaged startup, capture probing, embedded Web, helper identity/notices and clean-shutdown checks.
- Windows H.264 and VP8 Host gates passed live/paused settings, background recovery, source replacement, source-end/restart, decoded Viewer frames and complete resource cleanup.
- Final `0b370e6a` differs from the CI candidate only in TODO/status closure. Full Web/Go/native checks and website build passed again. The access/privacy gate passed credential-leak and cleanup checks.

## Impact / 影响

Compatible feature update for **v1.5.0**. No wire or storage version change. Older Apps retain their existing capture behavior through capability negotiation. Publication and matching deployments follow the release workflow.

The independent Windows x86 experiment is excluded. Physical 32-bit Windows, macOS/Linux capture, unsupported Windows border control, and actual border-permission denial remain outside this candidate's acceptance evidence. The original missing-window, public-link timeout and URL-handler crash reports still need affected-machine diagnostics.
This commit is contained in:
AAA authored and GitHub committed 2026-09-17 08:29:15 +08:00
1 parent cdd503a98d
commit d76e94eb71
52 files changed
+1380 -298
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

internal/app/app.go
+6 -5
View File
@@ -620,11 +620,12 @@ func discoverNativeMedia(ctx context.Context, configuredPath string) nativeRunti
captureProcess: path,
capture: capabilities,
capabilities: loopback.NativeMediaCapabilities{
Video: summary.Video,
ProcessAudio: summary.ProcessAudio,
SystemAudio: summary.SystemAudio,
HardwareH264: summary.HardwareH264,
SoftwareVP8: summary.SoftwareVP8,
Video: summary.Video,
CaptureBorderControl: summary.CaptureBorderControl,
ProcessAudio: summary.ProcessAudio,
SystemAudio: summary.SystemAudio,
HardwareH264: summary.HardwareH264,
SoftwareVP8: summary.SoftwareVP8,
},
}
}
internal/app/loopback/server.go
+6 -5
View File
@@ -62,11 +62,12 @@ type Health struct {
}
type NativeMediaCapabilities struct {
Video bool `json:"video"`
ProcessAudio bool `json:"processAudio"`
SystemAudio bool `json:"systemAudio"`
HardwareH264 bool `json:"hardwareH264"`
SoftwareVP8 bool `json:"softwareVP8"`
Video bool `json:"video"`
CaptureBorderControl bool `json:"captureBorderControl,omitempty"`
ProcessAudio bool `json:"processAudio"`
SystemAudio bool `json:"systemAudio"`
HardwareH264 bool `json:"hardwareH264"`
SoftwareVP8 bool `json:"softwareVP8"`
}
type Server struct {
internal/app/nativecapture/probe.go
+20 -17
View File
@@ -42,30 +42,33 @@ type Adapter struct {
}
type Capabilities struct {
Protocol int `json:"protocol"`
Platform string `json:"platform"`
PlatformBuild string `json:"platformBuild"`
VideoCapture bool `json:"videoCapture"`
ProcessAudio bool `json:"processAudio"`
SystemAudio bool `json:"systemAudio"`
SoftwareVP8 bool `json:"softwareVP8"`
Adapters []Adapter `json:"adapters"`
Protocol int `json:"protocol"`
Platform string `json:"platform"`
PlatformBuild string `json:"platformBuild"`
VideoCapture bool `json:"videoCapture"`
CaptureBorderControl bool `json:"captureBorderControl,omitempty"`
ProcessAudio bool `json:"processAudio"`
SystemAudio bool `json:"systemAudio"`
SoftwareVP8 bool `json:"softwareVP8"`
Adapters []Adapter `json:"adapters"`
}
type Summary struct {
Video bool
ProcessAudio bool
SystemAudio bool
HardwareH264 bool
SoftwareVP8 bool
Video bool
CaptureBorderControl bool
ProcessAudio bool
SystemAudio bool
HardwareH264 bool
SoftwareVP8 bool
}
func (capabilities Capabilities) Summary() Summary {
summary := Summary{
Video: capabilities.VideoCapture,
ProcessAudio: capabilities.ProcessAudio,
SystemAudio: capabilities.SystemAudio,
SoftwareVP8: capabilities.SoftwareVP8,
Video: capabilities.VideoCapture,
CaptureBorderControl: capabilities.VideoCapture && capabilities.CaptureBorderControl,
ProcessAudio: capabilities.ProcessAudio,
SystemAudio: capabilities.SystemAudio,
SoftwareVP8: capabilities.SoftwareVP8,
}
for _, adapter := range capabilities.Adapters {
if len(adapter.HardwareH264) > 0 {
internal/app/nativecapture/probe_test.go
+18 -1
View File
@@ -42,7 +42,24 @@ func TestSummaryDoesNotConflateVideoAudioAndEncoder(t *testing.T) {
}
summary := capabilities.Summary()
if !summary.Video || summary.ProcessAudio || !summary.SystemAudio ||
summary.HardwareH264 {
summary.HardwareH264 || summary.CaptureBorderControl {
t.Fatalf("summary = %+v", summary)
}
}
func TestCaptureBorderCapabilityRequiresCaptureAndExplicitSupport(t *testing.T) {
for _, supported := range []bool{false, true} {
payload := validProbe
if supported {
payload = strings.Replace(payload, `"videoCapture":true`, `"videoCapture":true,"captureBorderControl":true`, 1)
}
capabilities, err := decodeProbe([]byte(payload))
if err != nil || capabilities.Summary().CaptureBorderControl != supported {
t.Fatalf("capture border support = %v, error = %v", capabilities.Summary(), err)
}
capabilities.VideoCapture = false
if capabilities.Summary().CaptureBorderControl {
t.Fatal("border control was advertised without video capture")
}
}
}
internal/app/nativecapture/stream.go
+11 -7
View File
@@ -45,13 +45,14 @@ type CaptureTarget struct {
}
type VideoOptions struct {
Target CaptureTarget
Codec string
AdapterIndex uint32
EncoderIndex uint32
Profile VideoProfile
RestoreToken string
OutputGroups int
Target CaptureTarget
Codec string
AdapterIndex uint32
EncoderIndex uint32
Profile VideoProfile
RestoreToken string
OutputGroups int
ShowCaptureBorder bool
}
type EncodedVideoOptions struct {
@@ -232,6 +233,9 @@ func StartVideo(parent context.Context, executable string, options VideoOptions)
if err != nil {
return nil, err
}
if options.ShowCaptureBorder {
arguments = append(arguments, "--show-capture-border")
}
stream, err := startStreamWithEnvironment(parent, executable, arguments, environment)
if err != nil {
return nil, err
internal/app/nativecontrol/session.go
+10 -8
View File
@@ -214,9 +214,10 @@ func (session *Session) Handle(ctx context.Context, payload []byte) (result any,
"share", diagnostics.ID(request.ShareID), "sourceKind", request.Source.Kind, "audio", audio,
"adapterIndex", request.AdapterIndex, "encoderIndex", request.EncoderIndex)
err := host.ReplaceSource(ctx, nativecapture.VideoOptions{
Target: request.Source,
AdapterIndex: request.AdapterIndex,
EncoderIndex: request.EncoderIndex,
Target: request.Source,
ShowCaptureBorder: request.ShowCaptureBorder && session.capabilities.CaptureBorderControl,
AdapterIndex: request.AdapterIndex,
EncoderIndex: request.EncoderIndex,
}, audio)
if err != nil {
return operationFailure(envelope, err), nil
@@ -468,11 +469,12 @@ func (session *Session) startShare(
ShareID: request.ShareID,
CaptureProcess: session.captureProcess,
Video: nativecapture.VideoOptions{
Target: request.Source,
Codec: request.Codec,
AdapterIndex: request.AdapterIndex,
EncoderIndex: request.EncoderIndex,
Profile: profile.Video,
Target: request.Source,
ShowCaptureBorder: request.ShowCaptureBorder && session.capabilities.CaptureBorderControl,
Codec: request.Codec,
AdapterIndex: request.AdapterIndex,
EncoderIndex: request.EncoderIndex,
Profile: profile.Video,
},
Profile: profile,
EdgeCapacity: request.EdgeCapacity,
internal/app/nativecontrol/session_test.go
+68
View File
@@ -9,6 +9,7 @@ import (
"log/slog"
"os"
"path/filepath"
"slices"
"strconv"
"strings"
"testing"
@@ -35,6 +36,10 @@ func TestMain(tests *testing.M) {
}
func runQuietCaptureFixture(directory string) {
arguments, _ := json.Marshal(os.Args[1:])
if err := os.WriteFile(filepath.Join(directory, "arguments.json"), arguments, 0600); err != nil {
os.Exit(1)
}
var state nativehost.CaptureState
state.State, state.Codec = "starting", "vp8"
var adapter uint32
@@ -100,6 +105,69 @@ func runQuietCaptureFixture(directory string) {
<-closed
}
func TestCaptureBorderPreferenceFollowsSourceAndProfile(t *testing.T) {
for _, supported := range []bool{false, true} {
t.Run(strconv.FormatBool(supported), func(t *testing.T) {
directory := t.TempDir()
t.Setenv("PIIK_QUIET_CAPTURE_FIXTURE", directory)
executable, err := os.Executable()
if err != nil {
t.Fatal(err)
}
session := New(executable, nativecapture.Capabilities{VideoCapture: true, SoftwareVP8: true, CaptureBorderControl: supported}, false)
t.Cleanup(func() { _ = session.Close() })
handle := func(request any) {
t.Helper()
payload, err := json.Marshal(request)
if err != nil {
t.Fatal(err)
}
value, err := session.Handle(t.Context(), payload)
if _, failed := value.(requestFailedResponse); err != nil || failed {
t.Fatalf("capture request failed: %v, %#v", err, value)
}
}
assertBorder := func(expected bool) {
t.Helper()
data, err := os.ReadFile(filepath.Join(directory, "arguments.json"))
var arguments []string
if err != nil || json.Unmarshal(data, &arguments) != nil {
t.Fatalf("capture arguments missing: %v", err)
}
if slices.Contains(arguments, "--show-capture-border") != expected {
t.Fatalf("capture border intent lost: %v", arguments)
}
}
start := startShareRequest{Version: 9, ID: "request_start", Type: "start-share", ShareID: "share_123456",
Source: nativecapture.CaptureTarget{Kind: "display", SourceID: "1", Title: "Fixture"},
ShowCaptureBorder: true, EdgeCapacity: 1, Codec: "vp8",
Profile: qualitySettings{Resolution: "1080p", MaxFramerate: 30, MaxBitrate: 5_000_000, DegradationPreference: "balanced"}}
handle(start)
assertBorder(supported)
profile := start.Profile
profile.Resolution = "480p"
handle(updateShareRequest{Version: 9, ID: "request_update", Type: "update-share", ShareID: start.ShareID, Profile: profile})
session.mu.Lock()
done := session.updateDone
session.mu.Unlock()
if done != nil {
select {
case <-done:
case <-time.After(3 * time.Second):
t.Fatal("profile update did not settle")
}
}
assertBorder(supported)
for _, show := range []bool{false, true} {
handle(replaceShareSourceRequest{Version: 9, ID: "request_replace", Type: "replace-share-source", ShareID: start.ShareID,
Source: nativecapture.CaptureTarget{Kind: "display", SourceID: "2", Title: "Other fixture"}, ShowCaptureBorder: show})
assertBorder(show && supported)
}
handle(stopShareRequest{Version: 9, ID: "request_stop", Type: "stop-share", ShareID: start.ShareID})
})
}
}
func TestQuietHostUpdatePreservesControlAndCancelsCleanly(t *testing.T) {
for _, outcome := range []string{"complete", "stop", "disconnect", "first-input-timeout", "backpressure-stop"} {
t.Run(outcome, func(t *testing.T) {
internal/app/nativecontrol/wire.go
+21 -19
View File
@@ -42,17 +42,18 @@ type qualitySettings struct {
}
type startShareRequest struct {
Version int `json:"version"`
ID string `json:"id"`
Type string `json:"type"`
ShareID string `json:"shareId"`
Source nativecapture.CaptureTarget `json:"source"`
Audio bool `json:"audio"`
AdapterIndex uint32 `json:"adapterIndex"`
EncoderIndex uint32 `json:"encoderIndex"`
EdgeCapacity int `json:"edgeCapacity"`
Profile qualitySettings `json:"profile"`
Codec string `json:"codec"`
Version int `json:"version"`
ID string `json:"id"`
Type string `json:"type"`
ShareID string `json:"shareId"`
Source nativecapture.CaptureTarget `json:"source"`
Audio bool `json:"audio"`
ShowCaptureBorder bool `json:"showCaptureBorder,omitempty"`
AdapterIndex uint32 `json:"adapterIndex"`
EncoderIndex uint32 `json:"encoderIndex"`
EdgeCapacity int `json:"edgeCapacity"`
Profile qualitySettings `json:"profile"`
Codec string `json:"codec"`
}
type updateShareRequest struct {
@@ -64,14 +65,15 @@ type updateShareRequest struct {
}
type replaceShareSourceRequest struct {
Version int `json:"version"`
ID string `json:"id"`
Type string `json:"type"`
ShareID string `json:"shareId"`
Source nativecapture.CaptureTarget `json:"source"`
Audio bool `json:"audio"`
AdapterIndex uint32 `json:"adapterIndex"`
EncoderIndex uint32 `json:"encoderIndex"`
Version int `json:"version"`
ID string `json:"id"`
Type string `json:"type"`
ShareID string `json:"shareId"`
Source nativecapture.CaptureTarget `json:"source"`
Audio bool `json:"audio"`
ShowCaptureBorder bool `json:"showCaptureBorder,omitempty"`
AdapterIndex uint32 `json:"adapterIndex"`
EncoderIndex uint32 `json:"encoderIndex"`
}
type stopShareRequest struct {