139 Commits
Author SHA1 Message Date
ForgQi dd7bb7a9ee Refine auth policy and migration compatibility
Centralize authorization into a policy layer so route, field, and user-permission checks all use the same decision point. The frontend now revalidates /v1/me after 403s or dropped connections, and the users page fetches the real role/permission catalog from the backend instead of hardcoding role descriptions. This also tightens field-level protection for sensitive config/template data and avoids exposing user-management controls when auth is off.

The backend and migrations are updated to handle LF/CRLF checkout differences in SQL migration checksums, with compatibility reconciliation for older databases and safeguards to prevent silent drift of previously shipped migrations.
2026-09-24 19:44:35 +08:00
Noah e4fb9c01df feat(desktop): bundle ffmpeg and run external tools without console windows (#1735)
* feat(cli): locate ffmpeg via ffmpeg_path / host / PATH and start child processes without console windows

- New `tools` module in both crates. `tools::command()` starts external
  programs with CREATE_NO_WINDOW on Windows; every ffmpeg, streamlink,
  yt-dlp, ytarchive and hook shell process now goes through it.
- ffmpeg is looked up in this order: the new optional `ffmpeg_path`
  setting, then `ServeOptions::ffmpeg` (for an ffmpeg shipped by the
  embedding host), then `ffmpeg` on PATH. Unset keeps today's behaviour.
- `ffmpeg_path` is global (not overridable per streamer) and visible in
  the redacted configuration.

* feat(api): GET /v1/tools reports ffmpeg availability, version and license

Read-only, allowed for every logged-in role. Returns whether the ffmpeg
that would be used runs, where it came from (config / bundled / path),
the first line of `ffmpeg -version` (cached per program; failures are
retried) and the license derived from its configure flags. Errors say
what to fix.

* feat(desktop): bundle ffmpeg in the Windows installer, same BtbN build as Docker

- desktop-publish.yml downloads the BtbN win64 GPL static zip of the tag
  and version the Dockerfile pins (read by .github/scripts/ffmpeg-version.sh),
  checks its SHA-256 and ships only ffmpeg.exe, LICENSE.txt and a README
  with the source links as bundle resources (tauri.ffmpeg.conf.json),
  installed under ffmpeg\.
- The shell passes resource_dir()/ffmpeg/ffmpeg.exe to ServeOptions::ffmpeg
  when the file exists. The ffmpeg_path setting still wins; builds without
  the overlay keep using PATH.

* ci(desktop): attach the FFmpeg corresponding source and download notes to releases

- New ffmpeg-source job: FFmpeg at the pinned commit, the BtbN build
  scripts at the pinned tag, and the source of every library those
  scripts build (fetched by their own download.sh at the commits pinned
  in scripts.d/), plus a README and SHA256SUMS. Uploaded on release
  events only; manual runs just list the files.
- On release events, append .github/desktop-release-notes.md to the
  release body once: setup.exe is the recommended download, and how to
  get past the browser warning and SmartScreen while it is unsigned.

* fix(cli): only use CREATE_NO_WINDOW when biliup has no console

`biliup server` in a Windows terminal keeps sharing its console with the
programs it starts, as before: Ctrl+C and their output still reach the
terminal. Only a process without a console window (the desktop app)
starts them with CREATE_NO_WINDOW. The choice is logged at startup and
covered by a Windows-only test that runs itself as a child with and
without a console.

* ci(desktop): link to the upstream FFmpeg source instead of uploading it to releases

Drop the ffmpeg-source job that packed about 2 GB of FFmpeg and dependency
source into every release. The installer's ffmpeg/README.txt and the release
notes now point to the FFmpeg commit and the BtbN/FFmpeg-Builds tree of the
bundled build (full commit hashes, resolved at build time), whose scripts.d/
pins the version and source URL of every library. The Docker image uses the
same BtbN tag, so the same links cover it.
2026-09-24 09:50:42 +00:00
Noah 024254a288 fix(build): restore editable installs after stream_gears packaging change (#1734)
* fix(build): restore editable installs after stream_gears packaging change

Editable installs only put python-source (crates/stream-gears) on sys.path and skip include, so the root biliup/ package was unimportable after maturin develop / pip install -e . Add crates/stream-gears/biliup/__init__.py, which loads the root package under the same name. Wheels are unchanged: maturin only walks stream_gears/ and the include globs, which match the root biliup/ first.

* test(stream-gears): cover editable installs

test_wheel_ships_stubs only checks RECORD for regular installs, since an editable install's RECORD lists the .pth. A new test puts only python-source on PYTHONPATH and checks that biliup and biliup.__main__ resolve to the root package.

* build: move biliup/ and pyproject.toml into crates/stream-gears

Editable installs (`maturin develop`, `pip install -e`) only put
`python-source` on sys.path and skip `include`, so with `biliup/` at the
repository root they could not import it. With `pyproject.toml`,
`biliup/` and `stream_gears/` in the same directory, `biliup` goes in
through `python-packages`, which editable installs honour.

- `biliup/` is moved unchanged; the loader added earlier in this PR is
  removed because that path is now the real package.
- `readme` and `license` point at the repository root, so the wheel's
  METADATA and `licenses/LICENSE` stay the same.
- CI, Termux and the Dockerfile build with
  `-m crates/stream-gears/Cargo.toml`.

Ref #1732

* docs: point source install commands at crates/stream-gears
2026-09-24 09:30:04 +00:00
Noah 6c06b85b1c feat(desktop): run biliup server in-process, drop PyInstaller sidecar (#1731)
* feat(cli): let serve run on a host-bound listener with a host-owned shutdown

ServeOptions gains two optional fields for embedding hosts:

- listener: serve on an already-bound socket, so the host knows the
  actual port (e.g. after falling back to port 0);
- shutdown: stop when the given future completes instead of installing
  Ctrl+C / SIGTERM handlers, which would otherwise take those signals
  away from the host process.

The CLI path is unchanged apart from binding the port before opening the
database, so a busy port now fails before data/ is touched.

Ref #1721

* feat(desktop): run biliup server in-process, drop PyInstaller sidecar

The Tauri app now depends on biliup-cli as a library and runs
entry::serve on its own tokio runtime instead of spawning
binaries/biliup.exe:

- listens on 127.0.0.1:19159, or a free port when that one is taken;
  the window shows a bundled startup page and switches to the Web UI
  once GET / answers 200 (startup failures are shown on that page);
- data/, ds_update.log, cookies.json and recordings live in
  app_data_dir(); on first start, data/ left in the install directory
  by the old sidecar is copied over and the original is kept;
- closing the window stops the server and waits for it (up to 10 s);
  no expect() on exit;
- a second launch focuses the running window instead of starting a
  second server on the same data.

Removed: the greet command, the Vite + React scaffold under
tauri-app/src, the shell:allow-execute capability, tauri-plugin-shell,
tauri-plugin-opener, GBK decoding of sidecar stdout and the bundled
binaries/ resource. The Web UI itself is unchanged.

Ref #1721

* ci(desktop): build the Tauri app directly, remove biliup.spec

The Windows desktop workflow no longer builds a wheel, installs it and
runs PyInstaller: it builds the Web UI (embedded into biliup-cli by
rust-embed) and then the Tauri app with tauri-action. Renamed
pyinstaller-publish.yml to desktop-publish.yml.

On release it uploads to the release as before; a manual
workflow_dispatch run only builds and uploads the installers as an
artifact, without touching any release.

The Dockerfile used biliup.spec to tell whether it was built from a
checkout (otherwise it clones master); it now checks pyproject.toml.

Ref #1721

* ci(desktop): smoke-test the installer on manual runs

Only on workflow_dispatch: pretend an old install left data/ in the
install directory, install the NSIS package silently, start the app and
require GET http://127.0.0.1:19159/ to return 200, the database to be
created in the app data directory, the old data/ to be copied there and
kept in place, and the server to stop when the window is closed.

Ref #1721

* feat(desktop): keep data next to the exe unless it is on the system drive

On Windows the data directory is the install directory again, as with the
old sidecar, when the executable is not on %SystemDrive%. When it is, the
first start asks the user (tauri-plugin-dialog): the app data directory
(default), the install directory, or another folder. The choice is saved
as {"data_dir": ...} in desktop.json under the app config directory and
not asked again; a saved directory that cannot be used is reported on the
startup page instead of silently creating a new one. Closing the dialog
quits.

Other platforms have no drive letters and install into read-only
locations, so they keep using the app data directory (or data_dir from
desktop.json) without asking.

When the chosen directory has no data/ yet, data/ left by an older
version is copied there and kept in place. Besides the install directory,
Windows also looks where the old bbup-app installers put the app (their
registry keys and %LOCALAPPDATA%\bbup-app), since a new product name
installs into a new directory.

The data directory is resolved on a separate thread because the dialog
blocks. The migration helpers move to data_dir.rs unchanged.

Ref #1721

* chore(desktop): replace the template names with biliup

- productName bbup-app -> biliup (installers biliup_<version>_x64-setup.exe
  and biliup_<version>_x64_en-US.msi, NSIS default install directory
  %LOCALAPPDATA%\biliup, Start menu entry biliup)
- identifier com.tauri-app.app -> com.biliup.desktop (app data directory
  %APPDATA%\com.biliup.desktop, which is the default data directory
  offered on the system drive)
- Cargo package tauri-app -> biliup-desktop (executable
  biliup-desktop.exe, so it is not confused with the biliup CLI), lib
  tauri_app_lib -> biliup_desktop_lib, drop the template authors
- npm package name, workflow artifact name and the README download link

Tauri derives the NSIS uninstall/registry keys and the MSI upgrade code
from the product name, so the new installers do not replace an installed
bbup-app; both show up in Apps & features. The app copies data/ from the
old bbup-app install directory on first start (see the previous commit),
after which bbup-app can be uninstalled; its uninstaller does not delete
data/.

Ref #1721

* ci(desktop): bump the workflow actions to their latest majors

actions/checkout v4 -> v7, actions/setup-node v4 -> v7,
actions/upload-artifact v4 -> v7 (Node 24 runtimes; the other breaking
changes are about fork checkouts in pull_request_target, caching driven by
a packageManager field and single-file uploads, none of which this
workflow uses). Swatinem/rust-cache@v2 and dtolnay/rust-toolchain@stable
already track the latest release.

tauri-apps/tauri-action v0 -> v1. v1 updates the name and body of an
existing published release it finds by tagName with releaseName and
releaseBody, which are empty here and would wipe the release notes, so
the release job now passes releaseId from the release event instead;
with releaseId the action uploads to that release without looking it up.

Ref #1721

* ci(desktop): smoke-test the new data directory rules on manual runs

Replaces the manual-run smoke step from the earlier smoke-test commit and
can be dropped together with it. On the Windows runner:

- install the v1.2.8 bbup-app NSIS package into C:\old-bbup-app and leave
  a data/ there;
- install biliup into its default directory on the system drive (with an
  old data/ in it); both products stay installed side by side;
- first start shows the data directory dialog and does not start the
  server; UI Automation clicks the default button (the dialog text and a
  screenshot are kept as evidence), then GET / must return 200, the
  database and the copied data/ must be in %APPDATA%\com.biliup.desktop,
  the original kept and the choice saved in desktop.json;
- a second start must not ask again; a preset desktop.json pointing at
  another directory stands in for "choose another folder";
- reinstall on D: (or a subst drive when the runner has none): no dialog,
  database and log in the install directory, data/ copied from the old
  bbup-app directory found through its registry key;
- uninstalling bbup-app keeps its data/.

Ref #1721

* Revert "ci(desktop): smoke-test the new data directory rules on manual runs"

This reverts commit 860c45e89f8e5dab11468c83db7c623b7304e4dd.

* Revert "ci(desktop): smoke-test the installer on manual runs"

This reverts commit b3831f014a2b37f81b71c96e8229a6254bee1b9c.
2026-09-24 07:28:02 +00:00
Noah 3f4583b40d feat(stream-gears): ship stream_gears package with accurate stubs (#1732)
* feat(stream-gears): ship stream_gears package with accurate stubs

The stream_gears/ package (__init__.py, stub, py.typed) never made it into
the wheel: pyproject only listed python-packages = ["biliup"], so maturin
generated its own __init__.py and the stub drifted from the implementation
(7 of 16 upload lines, login_by_qrcode typed as bool, missing upload
parameters, pyobject.Segment/Credit not accepted by the Rust side).

- Move the package to the repo root next to biliup/ (maturin resolves
  python-packages relative to python-source) and set
  module-name = "stream_gears.stream_gears" for a mixed layout.
- __init__.py re-exports the native module, adds Segment = PySegment and a
  Credit TypedDict; pyobject.py is removed.
- Rewrite stream_gears.pyi from the implementation, including
  StreamGearsError and the proxy/file parameters.
- Give upload an explicit text_signature so its string and list defaults
  show up in inspect.signature/help() instead of Ellipsis.

Top-level names only grow: Credit and Segment are added.

* ci(desktop): keep PyInstaller off the source stream_gears package

PyInstaller puts the repo root first on its search path, so the source
stream_gears/ (which has no native module) shadowed the installed package
and the frozen biliup failed to import stream_gears.stream_gears. Remove
the source copy after installing the wheel.

* test(stream-gears): check package layout and run stubtest in CI

test_package.py covers the new top-level names, the native submodule,
__all__ staying in sync with the native module, py.typed and the stub in
the wheel, upload's text_signature against PyO3's argument parsing, and
mypy --strict on a valid and an invalid example.

CI runs python -m mypy.stubtest stream_gears against the installed wheel
so the stub cannot drift from the implementation again.

* build(stream-gears): drop unused danmaku-client and async-trait deps

Neither is referenced anywhere in crates/stream-gears. danmaku is still
built through biliup-cli, so this only trims the manifest and lockfile
entries.

* refactor(stream-gears): keep the package in crates/stream-gears via python-source

Point maturin's python-source at crates/stream-gears instead of moving
stream_gears/ to the repository root. python-packages is resolved under
python-source, so the root biliup/ package is shipped through include.
The wheel contents are unchanged.

* ci: drop workarounds for a root-level stream_gears package

With stream_gears/ back in crates/stream-gears, nothing at the repository
root shadows the installed package, so PyInstaller and stubtest no longer
need to avoid it.
2026-09-24 06:48:40 +00:00
Noah 67e49354ad fix(recorder): stop stream-gears losing the tail, default to mesio (#1733)
* fix(stream-gears): write the cached GOP on disconnect, flush before the segment hook

The httpflv loop kept the current GOP in `flv_tags_cache` and only wrote it
when the next keyframe arrived. On EOF, a 30 s read timeout or a parse error
the function returned and the cache was dropped, so every disconnect,
reconnect and stream end lost the last 1-10 s. The loop now runs in an inner
function; whatever way it exits, the complete tags in the cache are written
before the original result (including the error) is returned.

`FlvFile` / `TsFile` renamed the `.part` file and ran the segment hook before
the old `BufWriter` was flushed, so the hook (FileValidator, upload) could see
a truncated file, and a failed final flush (disk full) was swallowed. Both now
go through `finish()`: flush and check the error, then rename, then run the
hook. `create_new` returns the flush error; `Drop` logs it with `error!`.

Tests: the rolling-segment test now expects 120 keyframes on disk (was 119)
and checks the size seen by the hook equals the final size; new cases for a
stream that ends cleanly, a read that times out and a parse error; a TS hook
size test; a failed-flush test.

Ref #1726

* feat(recorder): default to mesio when no downloader is configured

Rooms without a `downloader` setting followed the plugin hint, which is
stream-gears for almost every platform. They now use mesio: its FLV output
carries `onMetaData.keyframes` and every segment starts at timestamp 0, so
players can seek; it also records HEVC / Enhanced-FLV and HLS fMP4, which
stream-gears cannot. Only the hint mapping in `core/live.rs` changes; an
explicit `stream-gears` keeps using stream-gears, and plugins that hint
streamlink / yt-dlp are unaffected.

`DownloaderRuntime::from_type` no longer has a catch-all arm. `ffmpeg-external`
and `ffmpeg-internal` used to fall into it and silently run stream-gears; both
now run ffmpeg (external segmenting, same as `ffmpeg`). The arm for
streamlink / yt-dlp / ytarchive, which need the live stream and are built in
`downloader_runtime`, is listed explicitly and logs a warning.

WebUI, README, sample configs and CHANGELOG now say mesio is the default and
list the behaviour differences: the container follows the source (HLS fMP4 is
saved as .mp4), files are written under their final name without `.part`, and
Douyu Wangsu reconnects leave a 2.6-5.8 s overlap between segments.

Ref #1726

* feat(mesio): pass segment duration and size to the segment callback

mesio's segment-complete callback reports the duration and byte size of each
closed file, but biliup only logged them and built `SegmentInfo` with the path
and index. `SegmentInfo` gains `duration_secs` / `size_bytes` (None for the
other downloaders) and mesio fills them in. The end-to-end mesio test checks
the reported size equals the file size on disk.

Ref #1726

* fix(recorder): parse segment_time the same way in every downloader

`segment_time` was read three different ways. stream-gears indexed
`parts[1]` / `parts[2]` directly, so "3600" or "01:00" panicked the download
task (and an unparsable hour silently became 1); mesio returned 0 for anything
that was not HH:MM:SS and silently recorded without time splits; ffmpeg got
the raw string and accepted both forms.

`parse_segment_time` follows ffmpeg's duration syntax (`[HH:]MM:SS[.frac]`
or plain seconds; minutes and seconds below 60 when colons are used), so all
three downloaders now agree. stream-gears and mesio share
`DownloadConfig::segment_time_limit`: 0 means no time split for both (mesio
already did this; stream-gears used to split at every keyframe), and a value
that still cannot be parsed logs a warning and records without time splits
instead of panicking or being ignored silently. The time-range clamp uses the
same parser, so "3600" is shortened near the window end like "01:00:00".

Ref #1726

* feat(webui): enter file_size as a number with an MB/GB unit

The config still stores bytes. The control converts with 1024 (the same
sizes Windows Explorer shows), keeps an untouched value byte-exact even
when it is not a round MB/GB, and re-picks the unit when the form value
is replaced from outside.

In the streamer override, clearing a value that was already set now
saves null (no size split for that streamer); before, Semi dropped the
emptied key and the old value stayed. Clearing something typed in the
same session still falls back to inheriting the global setting.

* fix(webui): keep the file_size unit beside the number in narrow modals
2026-09-24 06:24:17 +00:00
Noah bff8815be4 feat(cli): biliup season subcommands (#1730)
* feat(uploader): public season API handle and BV lookup for archive view

- `SeasonApi` becomes public: `BiliBili::season_api()` returns it, and a
  `#[doc(hidden)]` `SeasonApi::with_hosts` points it at other hosts so
  callers outside the crate (the CLI tests) can use a local fake server.
  The `BiliBili` struct itself is unchanged.
- `archive_view_by_vid(&Vid)` queries the public view endpoint with either
  `aid` or `bvid`, so a BV number does not need a separate lookup.

Ref #1721

* feat(cli): biliup season subcommands

`biliup season list | sections <season_id> | add <section_id> --vid … |
remove <episode_id> | sort <section_id> [episode_id…] [--reverse]`, all with
`--json`. `add` fetches cid and title through the public view endpoint.
`sort` puts the listed episodes first and keeps the rest in their current
order, since B站 requires every episode of the section in the request.

The logic lives in `season_cli.rs`; dispatch only gains one match arm in
`main.rs` and in the copy in `stream-gears/src/server.rs`.

Ref #1721

* docs: list season and user subcommands in README and SKILL.md

Ref #1721
2026-09-24 05:30:01 +00:00
Noah 55a260f9da fix(stream-gears): Python signatures, types and error semantics (#1729)
* fix(stream-gears): raise exceptions instead of failing silently in the Python API

- Add `stream_gears.StreamGearsError` (subclass of `RuntimeError`, so
  existing `except RuntimeError` keeps working). Errors that used to raise
  `RuntimeError` now raise it, with the full error-stack cause chain.
- `download` / `download_with_callback`: connection failures, an
  incomplete first packet, FLV errors mid-stream and HLS errors raise
  `StreamGearsError` instead of panicking (`PanicException` derives from
  `BaseException`) or only logging and returning. The FLV path calls
  `httpflv::parse_flv`, now public, so `httpflv::download` and server-side
  recording are unchanged.
- `download_with_callback` accepts any callable. A callback exception is
  logged when it happens and re-raised once the download finishes; if the
  download fails too, it becomes the `__cause__` of the `StreamGearsError`.
- `PySegment` fields are readable; `segment` also accepts a dict or an
  object with `time` / `size` attributes, and `desc_v2` items accept
  objects with `type_id` / `raw_text` / `biz_id` besides dicts.
- `upload`: malformed `extra_fields` raises `ValueError` instead of being
  dropped; empty or blank still means none.
- `login_by_sms` raises on failure instead of returning `False`; invalid
  `ret` JSON raises `ValueError`; tokio runtime creation errors raise
  instead of panicking.
- The seven login functions get `proxy=None` defaults (PyO3 >= 0.24 no
  longer defaults trailing `Option` arguments).

Ref #1721

* test(stream-gears): offline pytest cases for error semantics and argument types

Ref #1721

* fix(stream-gears): turn FLV parser panics in download into StreamGearsError

`httpflv::parse_flv` still `expect`s on malformed AAC/AVC/script tag data.
Catch the unwind at the binding boundary so `download` raises
`StreamGearsError` instead of `PanicException`, without changing the
parser that server-side recording shares.

Ref #1721

* test(stream-gears): update smoke suite for StreamGearsError and optional proxy

The login functions now default proxy to None, so the strict xfail cases
for it pass and are removed (test_errors.py covers the same assertion).
StreamGearsError joins the exported names; PySegment fields are readable.

* feat(stream-gears): optional cookie file path for login functions

`login_by_sms`, `login_by_web_cookies` and `login_by_web_qrcode` take a
`file` argument (default `"cookies.json"`, same as before) for where the
login info is written. `login_by_qrcode` takes `file=None` and writes the
login info there when given; it still returns the JSON string.

Ref #1721
2026-09-24 05:27:39 +00:00
Noah 753aade9f4 refactor(cli): single entry for native binary and main_loop (#1728)
* refactor(cli): single entry for native binary and main_loop

Move argument parsing, logging setup and command dispatch from
biliup-cli's main.rs into `biliup_cli::entry`, and make
stream_gears.main_loop call the same entry instead of its own copy.

- Both entries default to `server` when no subcommand is given (or the
  legacy `start`), expand `~` in paths, and log to `ds_update.log`
  (server only, no rotation) instead of `download.log` on the wheel.
- The log filter honours `RUST_LOG`; an explicit `--rust-log` overrides it.
- main_loop no longer calls process::exit on clap errors: `--help` and
  `--version` print and return, other parse errors raise SystemExit with
  clap's exit code. Logging uses try_init, so a second call in the same
  process does not panic.
- Export `entry::serve(ServeOptions)` so a host can run the Web server on
  its own tokio runtime.
- Document config_bindings as deprecated (behaviour unchanged).

Ref #1721

* test(stream-gears): cover the shared entry from the installed wheel

Exit codes of --help/--version/--bad-flag, main_loop called twice in one
process, and biliup server writing ds_update.log with RUST_LOG and
--rust-log precedence.

Ref #1721

* test(stream-gears): update smoke test comment now that --help no longer exits

* build(stream-gears): drop clap, unused after the shared entry

* ci(docker): start the built image and check GET / returns 200

The test job only built the image. Also run it with the default CMD and
require the Web UI to answer and ds_update.log to be written.

Ref #1721
2026-09-24 03:58:52 +00:00
Noah 8d482db4a6 feat(uploader): season (合集) API in biliup crate (#1727)
Add BiliBili::seasons, season_section, archive_view, add_to_season,
remove_from_season and sort_season_episodes, matching the endpoints and
payloads of bili_webup.py. Response types keep unknown fields in
`extra` so new fields from Bilibili do not break parsing.

Ref #1721
2026-09-24 03:55:02 +00:00
Noah 44df87f211 build(stream-gears): make crate cargo-testable, drop unused deps (#1724)
* build(stream-gears): enable pyo3 extension-module via default crate feature

Plain cargo build/test of stream-gears no longer links libpython, so it
works without Python dev files. build.rs adds the macOS cdylib link args
that maturin would otherwise pass. Wheel builds are unaffected: maturin
already enabled pyo3/extension-module through pyproject.toml.

Ref #1721

* build(stream-gears): drop 13 unused dependencies and biliup-cli python-bridge feature

bytes, ormlite, rust-embed, mime_guess, tower-http, sqlx, serde,
thiserror, futures, async-channel, url, chrono and fancy-regex have no
references under crates/stream-gears/src. The python-bridge feature of
biliup-cli is declared but never used by any cfg or dependent crate.

Ref #1721

* ci(test): run cargo test for the whole workspace including stream-gears

Ref #1721
2026-09-24 03:16:55 +00:00
Noah e83123b3be test(stream-gears): pytest smoke suite for the wheel in CI (#1723)
* test(stream-gears): pytest smoke suite for the wheel in CI

Offline checks against the installed wheel: exported names, UploadLine
members, PySegment/config_bindings basics, `biliup --version` and
`python -m biliup --help` in a subprocess. OPS-19 (login functions
require `proxy`) is recorded as a strict xfail.

The linux x86_64 job installs the abi3 wheel into a venv and runs the
suite before the free-threaded build.

Ref #1721

* chore(stream-gears): move manual scripts to examples/

Drop test_server.py (only printed the default config; now covered by
the smoke suite). Move login.py, test_upload.py and test_download.py to
examples/ so pytest never collects them, and take the video/cookie
paths and stream URL from argv/env instead of hard-coded E:\ paths.
Also pass proxy explicitly (OPS-19), use PySegment (OPS-20) and catch
ValueError from str.index (OPS-38) so the scripts run on this version.

Ref #1721

* test(stream-gears): drop internal bug id from xfail reason
2026-09-24 03:14:32 +00:00
Noah 364f2cf295 chore: remove dead Python/packaging leftovers (#1722)
* chore: remove unused setup.py, root .sqlx cache and stream-gears migration copy

setup.py is ignored by the maturin build backend. The workspace has no sqlx
compile-time query macros (only biliup-cli's migrate!()), so the root .sqlx
offline cache is never read, and stream-gears never runs migrations, so its
copy of 1_initial.sql is never executed.

Ref #1721

* chore(stream-gears): remove unconstructed RustDanmakuClient

danmaku.rs defined a RustDanmakuClient that was never constructed nor exposed
to Python; the live one is biliup_cli::server::core::downloader::RustDanmakuClient.

Ref #1721

* chore: drop unused quickjs install from Docker image and Termux wheel

Nothing imports quickjs any more (Douyu signing moved to Rust), yet the Docker
image still compiled it (pulling g++ just for that) and the Termux job injected
it into the wheel's dependencies.

Ref #1721

* build: point Homepage at biliup/biliup and align abi3 floor with requires-python

requires-python is >= 3.9 (yt-dlp and streamlink both require 3.9), but the
extension was built as abi3-py38, producing cp38-abi3 wheels. Bump to abi3-py39
so the wheel tag matches, and make the Termux sed that strips abi3 match any
minor version.

Ref #1721

* build: keep original Homepage URL

Per review, leave the Homepage metadata unchanged.

Ref #1721
2026-09-24 03:07:54 +00:00
Noah 53aeba193f fix(douyu): ffmpeg / mesio / streamlink / 边录边传也用上网宿 expire=0,HEAD 探测 + 403 退回原直链 (#1719)
* fix(douyu): ffmpeg / mesio / streamlink / 边录边传也用追加 expire=0 的网宿直链,拉流前探测 403 退回原直链

#1713 只让 stream-gears 用上追加 expire=0 的直链(它在首连里自带 403 兜底),
其它下载器仍拿原直链,录斗鱼网宿房间每 300 s 断一次。

改为在下载公共层(DownloadTask::download)交出直链前,用同一组请求头对追加过
expire=0 的网宿直链探一次:403 改用原直链,其余结果保留追加后的直链。只等响应头、
不读 body,不是追加直链时不发请求。stream-gears 仍走自己的 connect() 兜底。

* fix(douyu): expire=0 探测改用 HEAD,只有 2xx 才保留追加后的直链

实测网宿 token 一经 GET 成功即作废:GET 探测拿到 200 后,下载器用同一直链再连
8 次里 6 次只收到一个 GOP(0.2–2.3 MB)就被断开;HEAD 同样校验签名(改写 expire
即 403),之后 GET 4/4 正常持续。探测失败或返回非 2xx 时保守退回原直链。

* fix(streamlink): 直链型 runtime 每次拉流改用本次解析出的直链

streamlink runtime 在录制开始时固定了第一次解析的直链,重试时 check_stream 刷新的
DownloadConfig::url 用不上:斗鱼网宿 token 连过一次就作废,断开后每次重连只拿到一个
GOP,1 秒一个文件;也绕过了 expire=0 探测。runtime 的 url 就是插件解析出的流直链时
(无 runtime_options),改用 DownloadConfig::url。

* fix(douyu): 追加 expire=0 的直链探测通过却零字节失败时,本次录制余下的拉流改用原直链

HEAD 探测与下载器实际的 GET 若被网宿判得不一样(HEAD 200、GET 403),下载器会
反复失败直到重试用尽。DownloadTask 记一个粘性标记:非 stream-gears 下载器用追加
直链拉流、一个字节没写出且没有正常结束,之后本任务不再追加、也不再探测。
2026-09-23 15:01:53 +00:00
Noah d343386fbd feat: 用户管理与分级权限(超级管理员 / 操作员 / 只读观察者) (#1718)
* feat(web): multi-user accounts with fixed admin/operator/viewer roles

- migration 6 adds web_users and copies the existing single admin with
  the same id and password hash, so upgrades need no action and existing
  sessions stay logged in
- table-driven permission middleware on every business route (default
  deny), 403 for missing permission, 401 when logged out
- hooks/override and secrets redacted for non-admins; operators may only
  pick registered bilibili accounts for templates
- /v1/web-users management API, /v1/me and self password change
- login rate limit: 5 consecutive failures lock the username for 30s
- revoked sessions cut long-lived websocket/stream responses

Refs #1717

* feat(cli): add `biliup user list` and `biliup user reset-password`

Refs #1717

* feat(web-ui): user management page, account menu and editable login name

- /users page (super admin only): create users, change role, reset
  password, disable, force logout, delete; table on desktop, cards on
  mobile, with loading/empty/error states
- sidebar filters menu entries by the permissions from /v1/me and shows
  a no-access state for pages the role cannot open
- account menu in the sidebar footer with change password and logout,
  hidden when --auth is off
- 403 responses show a warning toast instead of redirecting
- login page username is editable (default biliup)

Refs #1717

* feat(web-ui): hide actions the current role cannot perform

Streamer add/edit/delete/pause, hook and override editing, template and
upload actions, bilibili account management and config saving are only
rendered for roles that have the matching permission. Account and type
tree requests are skipped for roles that would get 403.

Refs #1717

* fix(web-ui): wait for permissions on restricted pages, surface pause errors

- pages that need a permission not every role has render a spinner until
  /v1/me returns, so roles without it never fire requests doomed to 403
- user management is hidden when --auth is off, with an explanation if
  the page is opened directly
- the pause button goes through the shared response handler, so a 403
  shows the permission toast instead of failing silently
- permission errors sent as plain text keep their message

Refs #1717

* fix(web-ui): keep user actions reachable on tablet and narrow desktop

Refs #1717

* fix(py): handle the user subcommand in the stream-gears entry point

Refs #1717
2026-09-23 14:59:58 +00:00
Noah 3ab1a4630b feat: live-url 同一房间 5 秒去抖(失败重取带 fresh=1 绕过);斗鱼网宿直链追加 expire=0,不再每 5 分钟断流 (#1713)
* feat(preview): live-url 同一房间 5 秒去抖,播放失败后的重取带 fresh=1 绕过

每次打开预览都向平台 API 取一条新直链(带 cookie 时消耗登录态配额,B 站接口有频控)。同一房间
5 s 内的重复请求(StrictMode 双调用、连点、同一房间多个小窗)复用上一次结果并标 cached: true,
不再打平台;前端播放失败后的重取传 ?fresh=1 一定拿新 token(抖音某节点 404、直链到期都靠它)。
缓存插入时顺手清掉一分钟前的条目,表不会长。

Part of #1708

* feat(douyu): 网宿直链末尾追加重复的 expire=0,不再每 5 分钟断流切文件

斗鱼网宿 CDN 按直链里的 expire=300 每 300 s 关闭一次连接,stream-gears 录斗鱼因此
5 分钟一个文件。实测网宿的 wsAuth 只校验第一个 expire、连接寿命取最后一个:保留原值、
末尾再追加 &expire=0 能通过校验且不再断开;删掉或改写原值都会 403。

- douyu:只对网宿直链(host 为 ws*.douyucdn.cn 或 fcdn=ws)、且只有一个非 0 expire 时追加;
  火山(force_hs)、华为等其它线路与已是 expire=0 的直链不动。
- stream-gears:首连 403 且是追加过的直链时,本次改用原直链再连一次(同一 token 仍有效),
  之后照旧按 300 s 断开续录;每次拉流最多多一次请求,不进外层重试。
- 其它下载器(ffmpeg / mesio / streamlink / 边录边传)与预览直连没有这层兜底,照旧用原直链。
2026-09-23 11:06:06 +00:00
Noah 48dc8fd65b fix(preview): 中转模式在有 RTT 的链路上频繁缓冲——旁路挪到解析点、掉队原地续播、缓冲分低延迟 2 s / 流畅 5 s 两档 (#1716)
* fix(preview): stream-gears FLV 在解析点旁路,不等写盘循环整 GOP 落盘

httpflv 的写盘循环把 tag 攒在 flv_tags_cache 里,等下一个关键帧到来才把整个 GOP
一起写出——预览的旁路点原来就挂在这个写出循环里,于是预览端每个 GOP 间隔(B 站
常见 2–4 s)收到一次整 GOP 的突发、其余时间一个字节都不来,且整体落后一个 GOP。
播放器拿到的缓冲恰好在下一个突发到达时耗尽,零余量:同机 0 ms 下勉强连上,链路
上稍有 RTT / 抖动就 waiting;GOP 长于 mpegts.js 的追帧阈值时更是每个 GOP 都跳一次
再停一次(实测 GOP 4.2 s 的房间同机也有 85% 时间处于 waiting)。

把旁路挪到 tag 解析完、进缓存之前:预览按 tag 实时收到,与 mesio 路径(在条目进
管线前 tee)一致。写盘循环一个字节不变;旁路仍只是 push,不 await、不返回错误。

实测(arrival-probe,40 s):同一斗鱼房间直连 CDN 到达连续(最大静默 0.24 s),
中转修前每 2.0 s 一个 ~940 KB 突发、20 个静默间隙全部 > 1 s。

Refs #1712 #1713

* feat(preview): 快照保留最近 6 s 的 GOP,新订阅者起播就有几秒缓冲

原来快照只含「当前 GOP」:新订阅者从零到一个 GOP 之间的缓冲起步,之后到达 = 消耗,
缓冲永远这么薄,链路上几十到几百毫秒的抖动就 waiting。浏览器直连 CDN 时 CDN 会先
给几秒的 GOP 缓存,中转得给同样的深度。

PreviewSink 里多一个已完成 GOP 的队列(按关键帧到达时刻计,保留 SNAPSHOT_WINDOW = 6 s
以内的),快照 = 文件头 + 序列头 + 队列里的 GOP + 当前 GOP,实际深度在 6 s 到 6 s +
一个 GOP 之间。字节上限 MAX_SNAPSHOT_BYTES = 16 MB(6 s × 20 Mbps),超过先丢最旧的
GOP;单个 GOP 超过 MAX_GOP_BYTES 作废时历史一并放掉(不再连续);序列头内容变了或
fMP4 换了 init 时历史也放掉(旧 GOP 与新参数不配),内容相同的重发(stream-gears
分段时重放 onMetaData / 序列头)不影响。热路径只多几次 usize 比较与一次 VecDeque
push,仍不 await、不返回错误;无订阅者时同样保留(第一个订阅者也要立刻有缓冲),
每路多占约 6 s 的码率(1080p 2–5 MB)。

`with_snapshot_window(Duration::ZERO)` 给只关心分块判定的测试用。

Refs #1712 #1713

* fix(preview): 客户端掉队时从最近关键帧原地续播,不再断开重连

广播缓冲被覆盖(Lagged)时原来直接结束响应,浏览器看到「连接已断开,2 秒后重连」
的加载态,再花一次 HTTP 往返 + 等下一个关键帧才恢复;带宽长期偏紧的链路上就是
反复「触发加载」。

现在 hub 多一个 `resubscribe(previous, timeout)`:复用原订阅的连接许可(不占第二
个名额),向写入端再要一份从最近关键帧起的快照与新接收端;响应在同一条连接里续
发这份快照——FLV 去掉文件头(`Subscription::snapshot_after_header`,解码器不能中途
再收到文件头;序列头照给,掉队期间可能换过),fMP4 的 init segment 照发(MSE 接受
中途再来一份)。mpegts.js 的 remuxer 会把批次之间的时间戳跳变收平,画面表现为跳
过一段而不是转圈;一次响应最多 MAX_RESYNCS_PER_RESPONSE = 20 次,再多说明客户端
长期跟不上,结束响应交给播放器。每次重对齐 warn 日志带 skipped 与累计次数。

Refs #1712 #1713

* perf(preview): 客户端追赶积压时把广播缓冲里攒着的分块合并写出

实时转发每收到一个分块就写一个 chunked 帧(FLV 一个 tag 几百字节到几十 KB)。
客户端跟得上时缓冲里本来就只有一个,无从合并;追赶积压(起播快照刚发完、链路
抖动之后)时缓冲里攒着几十上百个,逐个写出是几十上百次 socket 写与 chunked 帧。
`coalesce` 在 `recv()` 到手后用 `try_recv` 把已经在缓冲里的后续分块合并成一个
响应块(上限 COALESCE_BYTES = 64 KB),缓冲空着就原样透传不复制,不引入任何等待;
合并途中撞上 Lagged 时把跳过数交回主循环按掉队处理,不吞掉。

Refs #1712 #1713

* fix(ui): 中转模式的播放器维持约 5 s 缓冲,用倍速而不是跳变追帧

mpegts.js 原来对中转、直连一律 liveBufferLatencyMaxLatency 3 s / MinRemain 0.5 s:
缓冲一超过 3 s 就跳到末尾前 0.5 s,等于把服务端刚给的几秒快照直接扔掉,之后只剩
0.5 s 余量,链路一抖就 waiting;GOP 长于 3 s 的流更是每个 GOP 跳一次再停一次。

按取流方式分两套参数(`Players` 新 prop `transport`,LivePreviewPlayer 按实际 source
传):
- relay:liveSync 倍速追帧——落后 > 9 s 用 1.1× 追到 5 s 附近,≤ 5 s 恢复 1×;
  只在落后 > 20 s(长时间暂停后)才直接跳到末尾前 5 s。与服务端 6 s 的快照窗口配合,
  播放器起播就有约 5 s 缓冲并一直保持。
- direct:沿用原参数(CDN 自带 GOP 缓存,ForgQi 实测异地直连不卡,不动)。

fMP4 的 MediaSource 播放器(只在中转路径上)同样改成:起播对齐到末尾前 5 s、
落后 > 9 s 用 1.1× 追、> 20 s 才跳;另加空洞跳过——服务端掉队重对齐后分片时间戳
前跳,缓冲区出现空洞,停在空洞前沿时跳到下一段起点,不再原地转圈。

Refs #1712 #1713

* style(preview): cargo fmt 与 clippy(len_zero)

* feat(preview): /live 支持 snapshot_ms,起播快照按播放器要的缓冲深度回溯

新订阅者拿到多深的快照,播放器就有多深的起播缓冲,也就是多长的中转延迟。之前一律给整个 6 s
保留窗口,只适合「流畅」档;低延迟档要 2 s 就只该拿 2 s,而不是拿 6–10 s 再用 1.1 倍速追 40 s。

- PreviewHub::subscribe_with_depth(depth):从当前 GOP 往前取,取到第一个关键帧到达时刻早于
  「现在 − depth」的 GOP 为止(含),起播缓冲落在 depth 到 depth + 一个 GOP 之间;Some(ZERO)
  只给当前 GOP,None 给整个窗口。Subscription 记住 depth,Lagged 重新对齐沿用。
- GET /v1/streamers/{id}/live?snapshot_ms=2000;不带参数行为不变。「开始直播预览」日志多一个
  snapshot_ms 字段。
- 保留窗口 SNAPSHOT_WINDOW 仍是 6 s,服务端内存不变;热路径只在有订阅请求时多一次
  VecDeque::make_contiguous(历史只有几个 GOP)。
- 单测 +2:深度切分的边界(零 / 跨线 / 大于全部历史 / 空历史)、带深度订阅与重对齐沿用深度。

Refs #1713 #1716

* feat(ui): 中转预览分「低延迟 / 流畅」两档,默认低延迟 2 s,卡了自动升档

ForgQi 在 #1713 下问「中转有办法不延迟 5 s 也不卡顿吗」。主因(整 GOP 成坨下发)修掉后 5 s 是
过度配置:同一条模拟链路(60 ms RTT ± 16 ms、丢包重传停顿、20 Mbps)上四路真实房间扫
1 / 1.5 / 2 / 3 / 5 s 目标缓冲各播 180 s,三路 FLV 从 1 s 起就 0 次 waiting;只有 HLS 来的
fMP4 要盖住一个分片(B 站 1–3 s),3 s 起 0 次、4 s 余量最低约 1–2 s。

- app/lib/live-buffer.ts:缓冲控制统一到一处(原来 mpegts.js 的 liveSync 管 FLV、fMP4 播放器
  自己一套 chase):起播跳到「末尾 − target」,快照落地那几秒里超了再对齐(慢链路上几 MB 要
  几秒才到齐,每对齐一次多看 2 s、最多 15 s),稳态超 max 倍速 / 超 hard 跳 / 跳过重对齐留下的
  空洞,稳态里缓冲耗尽一次报 onStall。档位:low = FLV 2 s / fMP4·TS 4 s,smooth = 5 s
  (目标与原 0d41140 相同,倍速阈值 9 → 7)。mpegts.js 在中转模式下关掉自己的追帧,直连一行未动。
- /live 请求带 snapshot_ms = 目标缓冲,起播就是目标深度,不再从 6–10 s 追回来。
- 低延迟档在稳态里卡一次 ≥ 1 s 或 60 s 内两次,本次播放升到流畅档重连(更深的快照起播),
  角标「卡顿,已切到流畅档」;换房间或改偏好即复位。
- 弹层标题栏与监视器工具栏各一个「低延迟 / 流畅」开关,记 localStorage biliup.preview.latency,
  默认低延迟;直连模式下不显示。

实测(同一模拟链路,180 s,hub 最新时间戳 → 已渲染):低延迟档三路 FLV 延迟中位 2.1–2.4 s、
fMP4 2.8–3.7 s,四路 0 次 waiting;流畅档 5.1 s。每 18 s 停 2.5 s 的链路上低延迟档卡两次后
自动升档,之后 130 s 0 次。

Refs #1713 #1716
2026-09-23 06:20:47 +00:00
Noah 486676db29 feat(ui): 直播预览阶段三——录制中房间实时码率折线图 (#1714)
* feat(api): GET /v1/live-rates 录制中房间的每秒写盘速率瘦端点

/v1/streamers 每次查全表、拼齐全部字段,10 s 一次的节奏不适合调快;码率曲线需要 1 s 粒度,
新加一个只读内存的瘦端点:扫一遍 worker,对每个 Working 的房间读一次阶段一 RateMeter 的窗口均值,
返回 [{id, bytes_per_sec, ts}]。与 /v1/streamers 的 live_bytes_per_sec 同源同值;不新增采样任务、
服务端不存历史、Cache-Control: no-store;注册在 router() 内,--auth 时同一道登录校验。

Part of #1708

* feat(ui): 实时码率数据层与 uPlot 折线组件

- app/lib/use-live-rates.ts:页面内唯一的一份采样源。有订阅者时每秒拉一次 /v1/live-rates,
  全部卸载 / 标签页隐藏即停;历史放模块级环形缓冲(每房间最近 3 分钟),组件卸载不清,
  切走再切回曲线还在;帧里没有的房间补 null 成断口。useSyncExternalStore 订阅,无新帧不重渲染。
- app/ui/RateChart.tsx:uPlot 折线,full(带时间轴 / Y 轴,单位按窗口最大值自动选 KB/s / MB/s,
  Y 从 0 起)与 sparkline(无坐标轴,hover 读数)两种形态;实例在 effect 里建 / 销毁,渲染期不碰 ref,
  StrictMode 双次 effect 不留多余 canvas;配色每次绘制时读 Semi CSS 变量,深浅色切换不重建。
  无采样 / 未录制只显示「—」占位,不画空图。
- app/ui/LiveRateChart.tsx:next/dynamic 懒加载入口 + 当前 / 峰值 / 均值统计条;
  uPlot 与其 CSS 打进独立 chunk(约 23 KB gzip),首屏不含。
- 新依赖 uplot ^1.6.32(本 PR 唯一新依赖)。

Part of #1708

* feat(ui): 三处接入码率折线——弹层可折叠折线、卡片与监视器 sparkline

- 卡片弹层底部新增「写盘速率 · 最近 3 分钟」可折叠区(默认展开,折叠态记 localStorage
  biliup.preview.rateChart):标题行带当前 / 峰值 / 均值,展开时才轮询与加载 uPlot。
- 控制台 / 直播管理页录制中卡片:名称行下一条 60 s sparkline(28 px,hover 显示时刻与数值)。
- 历史记录页「实时监视」:工具栏新增「码率图」开关(默认关,记 biliup.monitor.rateChart),
  开启后每路小窗下方一条 60 s sparkline;所有小窗共用同一次每秒请求。

Part of #1708

* style(api): live_rates 测试按 rustfmt 换行

* fix(ui): sparkline 的 hover 读数浮在折线上方,不再被容器裁掉

* docs(readme): 功能列表补充直播预览(封面 / 播放器 / 弹幕 / 码率曲线)

#1708 三个阶段至此完整,在 README 的功能列表里加一条说明。

Closes #1708

* feat(api): GET /v1/ws/live-rates 用 WebSocket 每秒推送码率帧

review 1(ForgQi):复用现有的 WebSocket 机制推送而不是每秒轮询。WebSocket 不占浏览器对同一主机
HTTP/1.1 的六个并发连接(监视器 4 路视频 + 1 条弹幕 SSE 已用掉五个,这正是当初不用 SSE 的原因),
与日志页 /v1/ws/logs 同款:Origin 与 Host 同源校验、进程内连接数上限(16)、Ping 回 Pong、Close 即止。
连上立刻发一帧,之后每秒一帧,形状与 GET /v1/live-rates 相同;HTTP 版保留给 WebSocket 连不上时回退与 curl 核对。
两个路由都在 router() 内,--auth 下同一道登录校验。单测用手写的最小 WebSocket 客户端走完握手 + 两帧,不引新依赖。

* feat(ui): 码率数据改走 WebSocket;卡片 sparkline 加开关并叠在封面上

review 1(ForgQi):
- use-live-rates.ts 首选一条 WebSocket /v1/ws/live-rates(服务端每秒推一帧,所有图表共用),
  一帧都没收到就断开(反代没放行 upgrade、401)时回退到每秒 GET /v1/live-rates 轮询;中途断开退避重连,
  标签页隐藏时断开、可见时重连;快照新增 transport: 'ws' | 'poll'。
- 控制台「进行中」区块头与直播管理页工具栏新增「码率图」开关(默认关,记 localStorage biliup.cards.rateChart,
  两页共用);开启后录制中卡片的 sparkline 叠在封面底部(深色渐变垫底、读数条在图区内;点击仍冒泡到封面打开预览),
  封面没有 / 加载失败时退到名称行下方。

* test(api): 码率 WebSocket 连接上限测试改用独立信号量

CI 上与同一二进制里并发跑的 WebSocket 推送测试争用进程级信号量(后者占着一个许可),
上限测试拿不满 16 个而失败;改为像 ws.rs 的日志连接测试那样传入独立的 Semaphore。
2026-09-23 03:44:43 +00:00
Noah ffa847e1c8 fix(preview): Twitch 不能直连的原因改为实测结论——清单服务器按 Origin 白名单放行 (#1715)
HLS 直连补测各平台清单 / 分片的跨域头:B 站 fMP4 的 m3u8 / init / .m4s 同域 ACAO *;抖音 TS 的主清单与
媒体清单 + 分片不同域(douyinliving.com → 100ycdn.com)都 *;虎牙 TS 同域、回显 Origin(清单轮询有
边缘节点间歇 403,靠重取 / 回落)。Twitch:usher 主清单无 ACAO,媒体清单 / 分片带 ACAO *,但清单
服务器对 Origin 做白名单——twitch.tv 与 localhost 200、127.0.0.1 / 其它站点 403,所以部署到自己域名
后浏览器直连必 403,仍标不能直连,原因改成这句。弹幕 SSE 两个测试改用各自的信号量,不再与并行
测试抢全局那把(偶发失败)。

Part of #1708
2026-09-23 03:42:37 +00:00
Noah 072604eed5 feat: 直播预览阶段二——内置播放器复用录制流(卡片弹层 + 历史页监视器) (#1712)
* feat(downloader): 直播预览 hub 与写盘旁路(httpflv / hls / mesio)

每路录制任务一个 PreviewHub(寿命 = 整场 execute(),跨分段与断流重试),下载器开始拉流时
attach 一个写入端,在写盘点旁边把同一份媒体字节 push 给它:httpflv 在关键帧写出循环、
stream-gears/hls 在分片 chunk 循环、mesio 在条目进入修复管线前。fMP4 分片流与 HEVC 序列头
标为不可预览;ffmpeg / streamlink / yt-dlp / 边录边传明确不可预览并给出原因。

写入端热路径只做三件事:GOP 缓冲追加、try_recv 订阅请求、broadcast::send。没有 .await、
没有锁等待、没有 ?;订阅请求由写入端在下一次推送时回应,快照(文件头 + 序列头 + 当前 GOP)
与随后的实时分块严格衔接。GOP 快照上限 8 MB,广播缓冲 256 槽 × 64 KB 且无订阅者时不占用,
每路订阅者数由信号量限制;慢订阅者收到 Lagged 后由端点断开让其重连。

Part of #1708

* feat(api): GET /v1/streamers/{id}/live 与列表的 preview 字段

新增 chunked 视频流端点:先发关键帧对齐的快照再接实时分块,Content-Type 按容器为
video/x-flv 或 video/mp2t,Cache-Control: no-store。房间不存在 / 未录制 404,下载器不能
旁路 415(带原因),连接数超限 429(每路 4、进程 16),尚未开始拉流或正在重连 503 +
Retry-After。慢客户端 Lagged 或写入端换代时结束响应,由播放器重连。注册在 router() 内,
--auth 时与 /v1/streamers 同一道登录校验。

GET /v1/streamers 每项新增 preview: { available, format: "flv" | "mpegts" | null, reason },
未录制为 null。

Part of #1708

* feat(preview): fMP4 分片流也能预览——init segment 快照、按分片广播、解出 codecs

mesio 的 HLS 会话拿到 fMP4 分片时不再标为不可预览:init segment(ftyp + moov)作为文件头
进快照,每个 moof + mdat 分片作为一个关键帧分块广播,新订阅者拿到 init + 最近一个完整分片,
端点 Content-Type 为 video/mp4,列表字段 preview.format 为 "fmp4"。

从 init segment 的 moov/trak/…/stsd 解出 RFC 6381 编码串(avc1.PPCCLL、hvc1.…、mp4a.40.N、
av01 / vp09 / ac-3 / ec-3 / opus / flac)放进 preview.codecs,供前端 MediaSource.isTypeSupported()
校验后 addSourceBuffer;解析只在 attach 后收到 init 时做一次,不在逐分块路径上。
stream-gears 的 HLS 路径没有下载 #EXT-X-MAP 初始化分片,遇到非 TS 分片时标为不可预览并说明原因。

按 ForgQi 在 #1708 的要求把 fMP4 纳入阶段二。

Part of #1708

* feat(ui): 直播间卡片的「预览」弹层——复用录制流的内置播放器

Player.tsx 加 isLive / muted / autoplay / type / codecs 与 onEnded / onError:flv 与 mpegts 走
mpegts.js(直播模式:不攒缓冲、追帧、自动清理 SourceBuffer),fmp4 走一个轻量 MediaSource
播放器(fetch ReadableStream → appendBuffer 串行队列,首个缓冲对齐播放位置,落后 > 3 s 追到
末尾,定期 remove 30 s 之前的缓冲),都以 Artplayer customType 挂载保持 UI 一致;
type 未知时用一次 GET 读 Content-Type 再起播。MediaSource.isTypeSupported 不通过时报错并显示
封面 + 原因,不转圈。

StreamerCard 录制中加「预览」图标按钮(不可预览时禁用,tooltip 显示后端给的原因),封面可点
即打开预览;弹层(LivePreview.tsx)关闭即卸载播放器、断开连接。服务端断开(掉队 / 换直链)
自动重连 3 次,之后交给用户手动重试;不可恢复错误 Toast 提示一次。零新依赖。

Part of #1708

* feat(ui): 历史记录页「实时监视」Tab——多路同屏监视器

<LiveMonitor /> 是独立组件:所有正在录制的直播间以小窗网格展示,可预览且在同屏路数内的
直接播放(默认静音),超出的显示封面、点击替换最早开始的一路(先进先出),下载器不能旁路
的显示封面与原因。同屏路数默认 DEFAULT_MONITOR_TILES = 4(可选 1/2/4/6/9,localStorage 记住),
每路小窗占对应直播间的一个预览连接。正在播放的列表由「用户选择 + 当前可预览房间 + 路数」
纯推导,不在 effect 里 setState。

历史记录页改为两个 Tab:「录制文件」(原表格)与「实时监视」,keepDOM=false 切走即卸载全部
播放器。要把监视器挪到别的页面只需换挂载点。

Part of #1708

* test(preview): 让旁路测试不依赖调度时机

rolling 测试先把订阅请求 poll 进写入端队列再开始写盘,源数据按 2 ms 一块限速送入而不是瞬间灌完
(瞬间灌完时订阅者必然掉队,那是慢订阅者测试覆盖的场景);慢订阅者测试只断言
「收到 + 被告知跳过 = 全部」与写入端不等待,不再对读取任务在负载下的吞吐做假设。

* fix(ui): mpegts.js 的 MSE 解码失败给出可读提示

典型来源是 TS 的 PES 没按访问单元对齐(如 Twitch 的 HLS),mpegts.js 拆出的帧不完整、浏览器拒绝
解码;原文 "Failed to execute 'appendBuffer'…" 对用户没有信息量,改为说明原因并强调录制不受影响。

* fix(preview): fMP4 与 TS 的快照从关键帧起,而不是从分段边界起

B 站 hls_fmp4 的分段(约 1 s)各含多个 moof/mdat 分片、且分段边界不对齐关键帧(实测 10 次冷启动
只有 1 次首帧是 I 帧):写入端现在把每个分段按 moof/mdat 切开,读 init 里的视频轨 ID 与
trex 默认 flags、trun 的 first_sample_flags / 首样本 flags / tfhd 默认 flags,视频首帧为同步样本的
分片才作为 GOP 起点。TS 分片起点则嗅探第一个视频 PES 的 NAL 类型(IDR/SPS → 起点,只有普通片
→ 追加到上一 GOP,说不准 → 按起点处理)。FLV 路径本来就按关键帧 tag 切,未变。

单测覆盖:分片切分(前导 styp、非同步 / 同步 / trex 默认 / 无视频轨)、跨分段边界的快照、
TS 嗅探(跨包的 PES、音频 PES 在前、非 TS 输入);另有 #[ignore] 的 B 站真实抓样核对。

* feat(danmaku): 预览播放器的实时弹幕——录制中的弹幕客户端旁路一份到 SSE

danmaku crate 的 RecorderConfig 新增可选的 live_tx(tokio broadcast):每解出一条事件,在写 XML
之前 send 一份(不等待,无订阅者即丢弃),XML 录制路径不变。DownloadTask 按 stream.danmaku
是否存在建一条 256 槽的广播,传给 danmaku_client;新增 GET /v1/streamers/{id}/danmaku(SSE,
每条 event 为 danmaku / gift / super_chat / guard_buy,data 为 {kind, text, name, color, ts},
进场消息不进;掉队跳过不断开;进程上限 32 路,超出 429;无弹幕客户端的平台 415)。
/v1/streamers 的 preview 字段新增 danmaku: bool,前端据此启用 / 禁用弹幕开关。

Part of #1708

* feat(ui): 预览弹层装载实时弹幕(开关控制),监视器默认关

Player.tsx 新增 danmaku: { url, enabled }:装 artplayer-plugin-danmuku(新依赖,Artplayer 官方弹幕
插件,5.3.0),开关打开时用 EventSource 连 /v1/streamers/{id}/danmaku,每条事件 emit 进弹幕层
(普通弹幕滚动、礼物 / 醒目留言 / 上舰置顶描边),关闭时断开 SSE 并隐藏弹幕层,不重建播放器。
弹层标题栏加「弹幕」Switch,默认开、记在 localStorage(biliup.preview.danmaku);平台没有弹幕
客户端时禁用并 tooltip 说明。监视器工具栏加同名开关,默认关(biliup.monitor.danmaku),
对全部小窗生效。localStorage 偏好统一走 useSyncExternalStore(app/lib/use-local-pref.ts),
同屏路数也改用它。

Part of #1708

* fix(preview): FLV 广播缓冲扩到 1024 槽,装得下 CDN 的整 GOP 突发

斗鱼 CDN 把整个 GOP(~4 s、~320 个 tag、~2 MB)一次突发送达,httpflv 解析器 1 ms 内推完;
刚建立的订阅者还没读到第一个实时分块,256 槽的缓冲就已被覆盖,收到 Lagged 被断开,重连后
又赶上下一个突发——冷启动 10 次只有 1 次播起来。FLV 分块是单个 tag,小而多,缓冲按格式分开:
FLV 1024 槽(实测均值 ~10 KB → ~10 MB),TS / fMP4 沿用 256 槽(分块大而少,16 MB 上限)。
回归测试:请求在关键帧回应后紧跟 600 个 tag 同步推完,订阅者仍逐个读到。

Part of #1708

* fix(ui): 弹幕开关禁用时说明是平台不支持还是未开对应 *_danmaku 配置

* feat(danmaku): 多个直播间的实时弹幕可复用一条 SSE(GET /v1/danmaku?ids=…)

浏览器对同一主机的 HTTP/1.1 并发连接只有 6 个:监视器同屏 N 路视频已占 N 个,弹幕若每路再开
一条 SSE,N=4 时就把列表轮询也饿死。新增 /v1/danmaku?ids=1,3,9 把若干路合成一条,事件 JSON
带 id 供前端分发;没有弹幕 / 未录制的 id 静默跳过,一个都没有时 404;一条最多 16 路,占 1 个
连接许可。原 /v1/streamers/{id}/danmaku 保留,走同一实现(事件同样带 id)。

Part of #1708

* feat(ui): 弹幕连接由页面持有并复用,监视器路数按 HTTP 协议限制

浏览器对同一主机 HTTP/1.1 只有 6 个并发连接,监视器 4 路视频 + 每路一条弹幕 SSE 就把列表轮询
饿死;9 路时也只有 6 路能播(实测)。改为 DanmakuFeed:页面(弹层 / 监视器)持有一条
/v1/danmaku?ids=… 连接,各播放器按 id 订阅;构造无副作用、open() 后有订阅者才连,StrictMode
下重复构造无害。同屏路数选项在 HTTP/1.1 下为 1 / 2 / 4,页面经 h2 / h3 加载时放开 6 / 9。
小窗弹幕字号 15px,弹层 22px。

Part of #1708

* fix(ui): 解码失败后关闭弹层必须断开拉流连接

<video>.error 非空时 mpegts.js destroy() 里清 SourceBuffer 的 remove() 抛 InvalidStateError,
后面的 transmuxer.close() 跑不到,连接留在服务端占着该路的预览许可直到刷新页面——真实 Twitch
TS 房间(mpegts.js 放不了)连开 4 次弹层第 4 次就 429。销毁前先把 MediaSource 从元素上摘下
(load() 使其 closed,mpegts.js 对 closed 的 MediaSource 跳过清理),destroy 仍抛则直接关
内部 transmuxer 兜底。

Part of #1708

* fix(ui): mpegts.js 换用可中止的自定义 loader,弹层关闭一定断开拉流

上一提交把 MediaSource 从元素上摘下只避免了 destroy() 抛异常,连接仍未断开:mpegts.js 自带的
FetchStreamLoader 在 Chrome 上处于 buffering 状态时 abort() 不调 AbortController.abort(),
而是等下一个 read() 回来再 reader.cancel()——解码出错后读取链已断,那一次永远等不到,连接留在
服务端占着该路许可(真实 Twitch TS 房间:关闭弹层后用 4 条 curl 探,1 条 429;再开一次 2 条 429,
直到浏览器退出才释放)。改为 customLoader:每次 open() 一个 AbortController,abort() 直接调它,
其余与原 loader 一致(直播不需要 Range)。

Part of #1708

* fix(ui): 忽略 Artplayer 在实例销毁后的自动重连,杜绝孤儿拉流连接

真正的泄漏来源:Artplayer 收到 video:error 后每隔 1 s 重设 art.url 最多 5 次(RECONNECT_TIME_MAX),
不检查自己是否已 destroy。弹层在解码失败时立刻卸载播放器,1 s 后 Artplayer 仍在已销毁的实例上
重设 url → customType 再建一个 mpegts.js 实例、再拉一路流,没有任何人持有它——每次打开泄漏
最多 5 条连接,直到刷新页面。mpegts / fMP4 两个 customType 入口对 art.isDestroy 直接返回;
fMP4 入口同时复用上一次的 dispose(同一实例重设 url 时先清掉旧的 MediaSource)。
mock 上用循环发送的 Twitch 坏样本复现:修前每开一次多 1 条连接(1→2→3),修后开关 4 次始终 0。

Part of #1708

* fix(ui): 自动播放重试前确认播放器还活着

* fix(preview): 最后一个订阅者走后释放广播缓冲

tokio broadcast 的缓冲只在被新数据覆盖时丢旧分块,而没有订阅者时写入端不再 send——最后一个
订阅者走后,缓冲里最后 N 个分块就一直挂着(FLV 1024 槽约 10 MB / 路;7 路真实短测中 RSS 在
一轮预览后从 95 MB 停在 117 MB)。写入端发现订阅者归零且缓冲里有数据时换一条新通道,旧缓冲
整个释放;之后的 subscribe() 拿新通道,对订阅者透明。

Part of #1708

* feat(preview): 直播预览取流方式配置(relay / direct)与浏览器直连的能力判定、直链端点

后端与浏览器不在一台机器时,浏览器直接向 CDN 拉一路能省服务器出口带宽(ForgQi,#1712 review 2)。
全局配置新增 preview_transport:relay(默认,复用录制流经 biliup 中转)/ direct(浏览器直连 CDN),
JSON 配置里可选字段,不需要迁移。/v1/streamers 的 preview 新增 direct: { capable, reason },按
review 1 实测的平台表判定:B 站 / 抖音 / 虎牙 FLV 能直连;斗鱼一 token 一连接、直连会挤掉录制;
Twitch CDN 无 ACAO;HLS(TS / fMP4)本版本回落——mpegts.js 放不了 m3u8,直连需 hls.js;其它平台
未验证按不能。新增 GET /v1/streamers/{id}/live-url:当前录制中那条流的直链、容器、平台与过期时间
估计(B 站 expires / 抖音 expire 十进制 Unix 秒,虎牙 wsTime 十六进制;斗鱼的相对秒数不算),
Cache-Control: no-store,与 /live 同在登录校验之内。DownloadTask 随每次 check_stream 刷新直链。

Part of #1708

* feat(ui): 直播预览取流方式——全局设置下拉,direct 模式下浏览器直连 CDN、失败自动回落中转

空间配置 → 全局设置新增「直播预览取流方式(preview_transport)」:经 biliup 中转(默认)/ 浏览器直连
CDN,文案说明两者区别与适用场景。LivePreviewPlayer 按(房间, 配置, 能力)派生取流方式:direct 且
平台能直连时先 GET /v1/streamers/{id}/live-url 拿直链(https 页面下 http 直链换 https),mpegts.js
以 cors / 不带凭据拉 CDN;出错或断开时重取一次直链再播,仍失败回落中转;平台不能直连的房间在
direct 模式下直接走中转。播放器左上角角标:「直连 CDN」或「已回落中转:<原因>」。监视器小窗共用
同一组件。异步结果与回落都带 sourceKey,配置切换时不在 effect 里同步 setState。

Part of #1708

* fix(ui): 直连模式下弹层脚注说明取流方式

* feat(preview): 直连模式向平台另取一条直链,不复用录制那条;斗鱼由此也能直连

ForgQi(#1712):走直连就不要复用链接,直接获取新链接。/v1/streamers/{id}/live-url 改为对该房间的
平台插件再调一次 check_stream,把新拿到的 raw_stream_url(新 token)给浏览器;平台返回未开播或
出错时 503,前端回落中转。实测新 token 的斗鱼 / 虎牙直链各读 25 s 稳定,同时录制那条连接无重试
——review 1 观察到的「斗鱼一 token 一连接」「虎牙第二条连接只给 GOP 缓存即断」都是同一直链两条
连接的问题,另取一条就绕开了。能力表:B 站 / 抖音 / 斗鱼 / 虎牙 FLV 能直连;Twitch 无 ACAO、HLS 需
hls.js 仍回落。Monitor 新增 PluginFor 消息按房间 URL 取插件。

Part of #1708

* fix(ui): 直连播够 60 s 后再断按直链到期处理,重取直链继续直连

斗鱼 CDN 的 token 每 5 分钟切一次连接(录制那条同样如此,日志里每 5:01 一次 StreamEnded → 重连),
B 站直链 1 h 过期。之前直连只允许重取一次,第二次到期就回落中转;现在区分「刚起播就失败」
(重取一次仍失败 → 回落)与「播了一阵才断」(视为到期,照常再取)。

Part of #1708

* feat(preview): HLS 直连——直链是 m3u8 时走 hls.js(按需加载),B 站 hls_fmp4 等也能直连

ForgQi(#1712):HLS 直连也支持一下。新依赖 hls.js ^1.7(动态 import,只在直连模式打开 HLS 房间时
才下载那一个 chunk,首屏包不变)。live-url 的 format 按新直链后缀给 flv / hls;direct_capability
只看平台的跨域放行,不再因容器排除——B 站 .m4s 分片的 CDN 同样 ACAO: *。Player 新增 hls 分支:
跟到最后 3 个分片、后向缓冲 30 s、分片请求不带凭据;清单 / 分片 404、CORS、解码等致命错误报给
LivePreviewPlayer 走重取直链 / 回落中转,清单变为非直播视为断开。Twitch 仍因无 ACAO 回落。

Part of #1708

* feat(preview): live-url 同一房间 5 秒去抖,播放失败后的重取带 fresh=1 绕过

每次打开预览都向平台 API 取一条新直链(带 cookie 时消耗登录态配额,B 站接口有频控)。同一房间
5 s 内的重复请求(StrictMode 双调用、连点、同一房间多个小窗)复用上一次结果并标 cached: true,
不再打平台;前端播放失败后的重取传 ?fresh=1 一定拿新 token(抖音某节点 404、直链到期都靠它)。
缓存插入时顺手清掉一分钟前的条目,表不会长。

Part of #1708
2026-09-22 16:25:00 +00:00
Noah 5ce14df240 feat(ui): 录制中房间显示封面、头像与实时码率(直播预览阶段一) (#1709)
* feat(downloader): 录制写盘字节计数与实时速率,/v1/streamers 新增 live_bytes_per_sec

各下载器在写出点往 `ByteCounter`(Arc<AtomicU64>)累加,写盘路径上只多一次
`fetch_add`,不加锁、不 await、不改任何控制流:

- stream-gears/httpflv:`FlvFile::write_tag` 按 11 + 数据 + 4 字节计
- stream-gears/hls:`download_to_file` 每个 chunk 落盘后计
- mesio:条目进入修复管线前按 `FlvData::size()` / `HlsData::size()` 计
  (writer 来自外部 crate,拿不到逐条写出量;修复只增删极少量 tag)
- ffmpeg / streamlink:子进程自己落盘,改为每秒读一次 `.part` 长度做差分

`DownloadTask` 持有 `RateMeter`:1 秒采样、10 秒滑动窗口取首尾差值。窗口明显长于
GOP,避免 httpflv 按关键帧批量落盘造成的抖动;采样任务随 `execute` 启停。

`GET /v1/streamers` 每项新增 `live_bytes_per_sec`(字节/秒):未录制、采样不足、
或下载器不经本进程写盘(边录边传、yt-dlp)时为 null。

Part of #1708

* feat(live): 解析主播头像、补斗鱼封面,新增录制中封面/头像的同源代理端点

`LiveStream` 新增 `avatar_url: Option<String>`:
- B 站:`getInfoByRoom` 同一次响应里的 `anchor_info.base_info.face`
- 抖音:房间对象 `owner.avatar_thumb.url_list[0]`(web / H5 两个接口都有)
- 斗鱼:betard 的 `owner_avatar`,缺失时退回 `avatar.{middle,big,small}`;
  封面 `room_pic` 一并补上(此前写死空串)
- 虎牙:`gameLiveInfo` / `liveData` 的 `avatar180`,与 screenshot 一样升到 https
- 其余平台为 None,不改行为

`DownloadTask` 在内存里保存当前录制的封面与头像地址(每次 check_stream 刷新,
随任务消亡,不落库、不加迁移)。`GET /v1/streamers` 每项新增 `live_cover_url` /
`live_avatar_url`(未录制为 null)。

新增 `GET /v1/streamers/{id}/cover` 与 `/avatar`:页面直接引用图片 CDN 会带上 biliup
的 Referer 被 B 站等拒绝(403),改由服务端以直播间页面地址作 Referer 拉取后转发,
按图片 URL 做 5 分钟内存缓存(上限 64 条、单图 4MB)。只按主播 id 取 worker 里已有的
地址,不接受任意 URL;上游不是图片时不转发。路由注册在 `router()` 内,`--auth` 时
与其它 /v1 接口同一道登录校验。

Part of #1708

* feat(ui): 录制中卡片显示直播间封面、主播头像与实时写盘速率

`StreamerCard`(主页「进行中」与直播管理网格共用)在状态为录制中时:
- 顶部加 16:9 封面缩略图,懒加载,图片走 `/v1/streamers/{id}/cover` 同源代理;
  加载中显示占位底色,加载失败整块消失,卡片回到原有样式
- 名称前加 22px 头像圆图(`/v1/streamers/{id}/avatar`),失败则不占位
- 时长右侧加写盘速率,按 B/s、KB/s、MB/s 自动换单位;后端未采样(null)显示「—」

图片地址带原始 URL 的短哈希作查询参数,换一场直播封面变了就不会沿用旧缓存。
未开播 / 暂停的卡片不变。直播管理列表视图新增「码率」列,主播名前带头像。

不引入新依赖:头像与封面用 next/image(项目已配置 unoptimized),速率格式化与
代理地址拼接放在 `use-dashboard.ts`。

Part of #1708

* perf(live): B 站封面/头像经代理时改取 CDN 缩略图

实测 B 站 `user_cover` 原图可达 11520×8640、4 MB,卡片上只显示成 16:9 缩略图和 22px
头像,原样转发既慢又占缓存。对 `*.hdslb.com` 的地址借 CDN 的 `@宽w_高h_1c` 后缀取
640×360 / 128×128 的裁切图(同一张图从 4 MB 降到约 25 KB);已带后缀或带 query 的
地址、以及其它平台的地址原样使用。

Part of #1708

* refactor(downloader): ffmpeg / streamlink 写盘速率改为解析 stderr 进度行,不再读盘

review 意见:每秒 stat 一次 `.part` 长度会额外占磁盘 IO。改为让子进程自己报告:

- ffmpeg 加 `-progress pipe:2 -nostats`,把 key=value 进度块打到 stderr(不受
  `-loglevel quiet` 影响),`total_size=N` 就是本进程累计写出字节
- streamlink 加 `--progress force`,stderr 不是终端时也按周期打
  `[download] Written 12.34 MiB to <file> (5s @ 2.50 MiB/s)`

都在既有的 `spawn_log` stderr 读取协程里顺带解析(`SubprocessProgress`),进度行降为
debug 日志不再刷 info;累计值只加增量、回退时重置基准。解析不出来只是没有速率,
不影响录制。删掉 `FileSizeProbe`。

Part of #1708

* feat(live): 主播头像开播时下载一次存到 data/avatar/,接口直接读本地文件

review 意见:头像几乎不变,不必每次经代理拉取。

- 新增 `common/live_image.rs`:图片抓取(带 Referer、大小与类型校验、B 站缩略图后缀)
  从 `api/live_media.rs` 挪到这里共用;新增 `AvatarStore`,按主播 id 存
  `data/avatar/<id>.<ext>` + `<id>.json`(记录来源地址与 Content-Type),先写临时文件
  再改名,同一时刻只允许一个下载写盘
- 开播进入录制流程时后台下载一次;后续 check_stream 拿到的地址变了才重新下载;
  下载失败保留旧文件、只记日志
- `GET /v1/streamers/{id}/avatar` 改为读本地文件(`Cache-Control: max-age=3600`);
  正在录制而本地还没有时现场下载一次再返回,失败退回旧文件;不在录制时只读本地,
  没有就 404。服务重启后凭 `<id>.json` 直接复用,不重复下载
- 封面仍走内存缓存的代理(每场直播都会换,落盘意义不大)

Part of #1708

* fix(downloader): streamlink 进度行打在 stdout,stdout / stderr 都解析

实测 `--output` 到文件时 streamlink 把控制台输出(含 `[download] Written …`)
打在 stdout 而不是 stderr,只解析 stderr 会得到恒为 0 的速率。两条流共用一个
进度跟踪器,避免重复累加。

Part of #1708
2026-09-22 06:34:11 +00:00
Noah 4e156ac683 fix(db): 启动时对齐被改写过的迁移校验和,1.2.4 的老库不再无法启动 (#1703)
* fix(db): 启动时对齐被改写过的迁移校验和,老库不再无法启动

1.2.5 就地修改了已经发布的迁移 2(#1684 把 SET tags = null 改成 SET tags = '[]')。
sqlx 会用文件内容的 SHA-384 校验每条已应用的迁移,于是 1.1.8 ~ 1.2.4 期间建立的
数据库升级后一律以 VersionMismatch 整体失败:进程退出码 1,容器反复重启。

在跑迁移之前加一步对齐:只有当 _sqlx_migrations 里记的校验和恰好等于登记在册的
历史摘要时,才在同一个事务里补跑新旧内容的差异(幂等 UPDATE)并改写校验和。
其余任何失配仍按原样报错,不会把用户自己改过的迁移悄悄放行。

* test(db): 钉死已发布迁移的校验和,防止再次就地改写

这次故障的根因是改了一条已经发布过的迁移。加一条断言把迁移 1..5 的 SHA-384 写死:
再有人就地改动已发布的迁移,CI 当场失败,而不是等用户升级时容器起不来。
2026-09-21 02:48:33 +00:00
ForgQi 9d02856a50 Integrate mesio (rust-srec) downloader
Add an in-process mesio downloader using the rust-srec mesio-engine. Introduces crates/biliup-cli/src/server/core/downloader/mesio.rs implementing FLV/HLS pull, timestamp fixes, keyframe indexing, and size/duration-based segmentation with pipeline_common, flv_fix and hls_fix. Wire Mesio into downloader enum/runtime and add UI options and public config docs. Update crates/biliup-cli/Cargo.toml and workspace Cargo.toml/Cargo.lock to include new dependencies and bump version to 1.2.5. Include tests for protocol selection, pipeline behavior and an e2e FLV download. Small helper changes: config enum docs and segment handling.
2026-09-21 00:24:52 +08:00
lplalbertandlplalbert 5ec63167d7 fix(douyu): remove incorrect video_loop check that skips live streams (#1700)
The Douyu API now returns videoLoop=1 for normal live streams,
causing the video_loop != 0 check to incorrectly filter them out
as looped/replay content. Remove this check and rely solely on
show_status to determine if the room is live.

Co-authored-by: lplalbert <“lplalber111@gmail.com”>
2026-09-20 14:52:24 +00:00
Noahandransxd dfb933f066 fix(douyu): tolerate empty PlayResponse data and use server Date for tt (#1698)
Douyu getH5PlayV1 validates tt against server time; local clock skew
returns error -9 with data:"". PlayResponse could not deserialize the
empty-string data, so the -9/-5 branches never ran. Parse Date from
getEncryption for tt (cache offset for reuse) and deserialize data via
Value so real error/msg surfaces.

Fixes #1680

Co-authored-by: ransxd <ransxd@users.noreply.github.com>
2026-09-14 16:48:13 +00:00
Noah 970dfe8b37 feat(upload): support first-class tid_v2 partition id (#1699)
Add optional tid_v2 across Studio, config/DB, CLI, Web UI, stream-gears,
and Python uploaders so new Bilibili partition IDs can be set alongside
legacy tid. Fixes #1662; related to #1562.
2026-09-14 16:19:53 +00:00
Noahandransxd 2295ae1eba fix(streamlink): httpstream prefix for FLV and Error backoff (#1682) (#1695)
Prefix already-resolved progressive HTTP(S) media URLs (e.g. Huya/Douyu
.flv CDN links) with httpstream:// so Streamlink's http plugin matches
them, while leaving HLS/DASH and plugin webpage URLs unchanged.

When a download returns Error (or Err) while the room is still Live,
keep exponential retry backoff instead of resetting retry_count to 0,
which previously caused "Retrying download in 0ns..." spam. Immediate
retry remains only for successful segment completion / clean stream end.

Co-authored-by: ransxd <ransxd@users.noreply.github.com>
2026-09-14 16:18:15 +00:00
Noah f85e66b6c2 fix(record): 文件名含点时不再截断扩展名或死循环 (#1687)
* fix(record): 文件名含点时不再截断扩展名或死循环

Path::with_extension 会把 Mr.Beast 变成 Mr.flv,冲突检测永远命中已有文件,generate_filename 的 loop 占死 tokio 线程且无法 stop。

* test(record): 合并 path_with_suffix 测试到已有 tests 模块

util.rs 已有 mod tests,新增第二个会 E0428。
2026-09-14 02:57:48 +00:00
Noah 1b426721d6 fix(upload): 定时发布把延迟秒数换成 Unix 时间戳 (#1686)
Web UI 存的是延迟秒数,B 站接口要 10 位时间戳。Python 版会做 time()+dtime,Rust 投稿路径漏了换算,定时稿被拒后边录边传首 P 会停摆。
2026-09-13 15:40:17 +00:00
Noah 9445db636b fix(danmaku): 虎牙弹幕从用户结构体读取昵称和正文 (#1689)
MessageNotice 的 tag 2 是 lSid 不是用户名。旧解析把 lSid 当字符串读,lSid 非 0 时连正文都对不齐,弹幕几乎全部丢弃。
2026-09-13 15:36:15 +00:00
Noah 8437b028c3 fix(db): 迁移 2 将空 tags 写成 [] 而不是 NULL (#1684)
uploadstreamers.tags 是 JSON NOT NULL。SET tags = null 会让从 Python 升级且 tags 为空字符串的实例卡在迁移 2。
2026-09-13 15:33:12 +00:00
Noah 1a56242696 fix(log): 下载器命令行日志脱敏 Cookie 和 OAuth (#1688)
* fix(log): 下载器命令行日志脱敏 Cookie 和 OAuth

ffmpeg/streamlink/yt-dlp/ytarchive 会把 Cookie 头和 auth-token 打进 ds_update.log,该文件可经 Web 日志页与 /static 下载,排障贴日志即泄露登录态。

* fix(log): 头部值整段脱敏并合并测试到已有 tests 模块

Cookie/Authorization 的值吃到下一个引号,OAuth token 不再漏出;新增 --xxx-password 形式 argv 的脱敏;sid 不再当密钥以免误伤斗鱼直链。util.rs 已有 mod tests,第二个会 E0428。
2026-09-13 15:31:43 +00:00
Noahandransxd 86bcaffd2a fix(sync): AUTO 测速超时与 file_size 覆写占位 null (#1693)
- AUTO 线路探测为列表请求与单线路测速增加 15s 硬超时,失败只跳过该线路
- POST 探测体按服务端 probe.post(默认 0.1MB)发送,避免固定 10MB 拖慢选线
- ConfigPatch 序列化跳过未设置的 file_size,避免落库/回传把全局值清成 None
- 迁移清理历史 override 中的 file_size: null 占位
- 同步保留边录边传管线健壮性修复(取消保留分段、错误提示、分段顺序等)

Co-authored-by: ransxd <ransxd@users.noreply.github.com>
2026-09-13 15:29:12 +00:00
Noah c4dbf81c5d fix(danmaku): B站数据包解析不再对畸形输入 panic,解码 panic 不再终结录制 (#1678)
decode_packet 只校验 packet_len 是否越界,未校验其不小于 16 字
节包头:远端发来 packet_len < 16 的畸形帧时,body 反向切片直接
panic;panic 发生在 tokio::spawn 的录制任务里,run() 的重连逻辑
只处理 Err,任务被杀死后没有任何重启机制,弹幕录制静默永久停
止,XML 文件也不会 finalize。

修复:
- packet_len < 16 视为不可信帧,丢弃剩余数据并保留已解析的包,
  同时消除 packet_len=0 时 offset 不前进的死循环隐患
- 压缩包递归解码增加深度上限(真实协议仅一层嵌套),恶意多层
  嵌套不再导致栈溢出
- 客户端在 WebSocket 与 TCP 两条路径上用 catch_unwind 包裹
  decode_message:任何平台解码器 panic 都降级为可跳过的解码错
  误,连接与录制继续存活

新增 5 项回归测试:零长度帧、短长度帧、合法帧后跟畸形头、单层
zlib 嵌套仍可解码、超深嵌套被丢弃。
2026-08-30 15:03:59 +00:00
Noah 631e1160b1 fix(flv): 使用 write_all 保证 FLV 标签与 JSON 输出完整写入 (#1676)
flv_writer 中三处使用 io::Write::write 写入数据:write 允许部分
写入且把已写字节数作为 Ok 返回,短写时 previous_tag_size 等
字段会静默缺字节,产出的 FLV 标签结构损坏,播放器与解析器
在该点之后无法继续解析。

将 write_tag 的 previous_tag_size 写入、write_previous_tag_size
与 to_json 的换行写入全部改为 write_all,公开签名保持不变。
新增短写 Writer 回归测试,验证 4 字节 previous_tag_size 在部分
写入场景下仍被完整写出。
2026-08-30 14:54:09 +00:00
Noah 2c3592f117 fix(bilibili): 稿件分页查询复用登录客户端,不再每页新建 Client (#1675)
archives() 之前在分页循环中每请求一页都重新 build 一个
reqwest::Client 和 cookie jar:连接无法复用,代理配置丢失导致
查询绕过 --proxy 直连;且 Jar::add_cookie_str 会把 "; " 拼接的
整串 Cookie 当作单条 Set-Cookie 解析,实际只有第一个 Cookie
生效,其余键值被当作 Cookie 属性丢弃。

改为复用登录时构建的 self.client:cookie store 已含全部登录
Cookie,UA 与原实现一致,并保留代理配置与连接池,同时按原
行为为该请求保留 60 秒总超时。
2026-08-30 14:52:47 +00:00
Noah 5c6c720fdc fix(upload): 测速时单条线路失败不再中断整体测速 (#1674)
Probe::probe 对每条线路的 ping 使用 ? 传播错误,任意一条线路
DNS/连接/读取失败都会让整个测速立即返回错误,即使后续线路
完全可用,调用方也会因 unwrap_or_default 直接退回默认线路。

现在单条线路网络错误或异常状态码只记录警告并跳过该线路,
继续测速其余候选;全部线路失败时返回统一错误。
2026-08-30 14:51:09 +00:00
NoahandNoah 616432d190 feat(sync): 恢复边录边传下载器的流式上传管线 (#1673)
Rust 重写后 sync-downloader 被静默映射为 stream-gears 落盘下载,
边录边传完全失效(#1612)。本次恢复 ffmpeg stdout 到 UPOS 流式
分片上传、逐段投稿/追加分 P 的完整链路,并修复可靠性问题:

- UPOS 拆分为上传 parts 与显式 complete 两阶段,parts 按
  partNumber 排序并校验连续,complete 带状态检查与重试
- 录制同时写临时文件(有界通道背压);短流/预传失败时按实际
  长度回退为文件重传,投稿成功后清理临时文件
- SyncSession 跨拉流重连保留 aid,分 P 严格按录制顺序提交,
  首投结果不确定时停止自动重试避免重复建稿
- 上传/投稿错误向上传播;外层重试只在分 P 推进后重置退避
- 修复 www.douyin.com/live/{id} 被解析成 web_rid=live 的问题

Co-authored-by: Noah <17682131+ransxd@users.noreply.github.com>
2026-08-30 14:50:37 +00:00
yjywybb 051f5c7eb0 fix(upload): preserve filename-invalid characters in submission titles (#1671) 2026-08-25 07:50:23 +00:00
ForgQi 03b7a84f55 Allow remote Web UI bootstrap with auth
This change updates the server default bind to 127.0.0.1 for safer local-only access while documenting how to expose the Web UI via --bind 0.0.0.0 --auth for remote or Docker use. It also removes the previous restriction that blocked first-time admin bootstrap on non-loopback addresses, allowing authenticated remote initialization and bumping the workspace version to 1.2.4.
2026-08-24 01:13:29 +08:00
NoahandNoah c351c39771 fix(server): 会话 Cookie 不再强制 Secure,修复 HTTP 远程访问登录后无法保持登录态 (#1669)
Co-authored-by: Noah <17682131+ransxd@users.noreply.github.com>
2026-08-23 15:20:11 +00:00
NoahandNoah 6cef8afe22 fix(stream-gears): 登录函数释放 GIL 后再执行阻塞网络请求 (#1667)
Co-authored-by: Noah <17682131+ransxd@users.noreply.github.com>
2026-08-22 19:25:48 +00:00
ForgQi eb8699b203 Add recording policy and time-range checks
This change adds a recording policy layer that enforces time-window eligibility and excluded title keywords before probing and before recording. It updates the streamer API to surface reasons like OutOfSchedule and TitleExcluded, clamps ffmpeg/segment durations to the configured window, and exposes the last rejection reason from the worker state for the frontend.
2026-08-22 19:15:48 +08:00
cls3389 8fe338b5f5 fix(douyin): 下播后回放房间仍被误判为直播中 (#1663)
抖音主播下播后,直播间跳转 reflow 回放页,接口仍返回 status=2
和可拉取的回放流,导致 monitor 循环误判开播并反复录制垫片流。

两处独立防线:
- get_room_info: finish_time > 0(抖音已写入直播结束时间)判定未开播
- select_stream_url: 流 URL 含 rtm_expr_tag=reflow_room_info
  (回放流标记)时拒绝作为直播流返回

Fixes #1532
2026-08-22 08:39:50 +00:00
NoahandNoah 3750ced583 fix(hls): respect playlist refresh lifecycle (#1665)
Co-authored-by: Noah <17682131+ransxd@users.noreply.github.com>
2026-08-22 08:37:05 +00:00
Noahandransxd e1c4b6887e fix(downloader): 纯视频流分段时避免序列头 expect 导致的 panic (#1664)
分段重建逻辑此前对 on_meta_data / aac_sequence_header / h264_sequence_header 使用 expect。纯视频流(无音频,aac_sequence_header 恒为 None)在首次分段时必然 panic,导致 7x24 录制任务中断。改为缺失时跳过并告警,不再中断录制。

新增回归测试 pure_video_stream_segments_without_panic:构造仅含 onMetaData 与 H264 序列头、无音频标签的 FLV 流并触发分段;修复前该测试会 panic(aac_sequence_header does not exist),修复后通过。

Co-authored-by: ransxd <ransxd@users.noreply.github.com>
2026-08-22 08:35:19 +00:00
Mostly_Harmless b438d72415 feat(web): 完善账号管理、稿件查询与安全边界 (#1660)
启动 Web 服务时自动登记 CLI 指定的 B 站凭据,并为账号登记、刷新和删除统一补上并发去重、原子写入及引用检查。

同时收紧默认监听、管理员初始化、认证路由、静态文件和日志 WebSocket,恢复 server 的 ds_update.log,并避免凭据及上游响应进入日志。

新增按账号查询远程稿件的 API 与 Web 页面,补齐严格分页和响应校验。

关联 biliup/biliup#1624

关联 biliup/biliup#1658

关联 biliup/biliup#724
2026-08-16 14:05:20 +00:00
vector0902androotuc ccb016be4f fix(upload): include lock file path in rate-limit lock error message (#1661)
Co-authored-by: rootuc <rootuc@notunixchan>
2026-08-15 06:40:18 +00:00
dreammu 3e8b55c2c4 fix(upload): update upload lines (#1656) 2026-08-07 09:28:12 +00:00
Micuks adf6a1c03b fix(hls): 选取最高画质的流,而非固定取 variants[0] (#1649)
HLS 下载器一直取 master playlist 里的第一个变体。Twitch 会把各档
转码清晰度排在源流前面,所以经常录到较低画质的转码流,而不是源流。

改为选取带 RESOLUTION、且码率最高的变体,并跳过 I-frame(trick-play)
流。若没有变体带分辨率,则回退到码率最高的非 I-frame 变体,最后回退
到第一个变体,保证在异常 playlist 下行为不退化。
2026-07-14 14:35:35 +00:00