Centralize authorization into a policy layer so route, field, and user-permission checks all use the same decision point. The frontend now revalidates /v1/me after 403s or dropped connections, and the users page fetches the real role/permission catalog from the backend instead of hardcoding role descriptions. This also tightens field-level protection for sensitive config/template data and avoids exposing user-management controls when auth is off.
The backend and migrations are updated to handle LF/CRLF checkout differences in SQL migration checksums, with compatibility reconciliation for older databases and safeguards to prevent silent drift of previously shipped migrations.
* feat(cli): locate ffmpeg via ffmpeg_path / host / PATH and start child processes without console windows
- New `tools` module in both crates. `tools::command()` starts external
programs with CREATE_NO_WINDOW on Windows; every ffmpeg, streamlink,
yt-dlp, ytarchive and hook shell process now goes through it.
- ffmpeg is looked up in this order: the new optional `ffmpeg_path`
setting, then `ServeOptions::ffmpeg` (for an ffmpeg shipped by the
embedding host), then `ffmpeg` on PATH. Unset keeps today's behaviour.
- `ffmpeg_path` is global (not overridable per streamer) and visible in
the redacted configuration.
* feat(api): GET /v1/tools reports ffmpeg availability, version and license
Read-only, allowed for every logged-in role. Returns whether the ffmpeg
that would be used runs, where it came from (config / bundled / path),
the first line of `ffmpeg -version` (cached per program; failures are
retried) and the license derived from its configure flags. Errors say
what to fix.
* feat(desktop): bundle ffmpeg in the Windows installer, same BtbN build as Docker
- desktop-publish.yml downloads the BtbN win64 GPL static zip of the tag
and version the Dockerfile pins (read by .github/scripts/ffmpeg-version.sh),
checks its SHA-256 and ships only ffmpeg.exe, LICENSE.txt and a README
with the source links as bundle resources (tauri.ffmpeg.conf.json),
installed under ffmpeg\.
- The shell passes resource_dir()/ffmpeg/ffmpeg.exe to ServeOptions::ffmpeg
when the file exists. The ffmpeg_path setting still wins; builds without
the overlay keep using PATH.
* ci(desktop): attach the FFmpeg corresponding source and download notes to releases
- New ffmpeg-source job: FFmpeg at the pinned commit, the BtbN build
scripts at the pinned tag, and the source of every library those
scripts build (fetched by their own download.sh at the commits pinned
in scripts.d/), plus a README and SHA256SUMS. Uploaded on release
events only; manual runs just list the files.
- On release events, append .github/desktop-release-notes.md to the
release body once: setup.exe is the recommended download, and how to
get past the browser warning and SmartScreen while it is unsigned.
* fix(cli): only use CREATE_NO_WINDOW when biliup has no console
`biliup server` in a Windows terminal keeps sharing its console with the
programs it starts, as before: Ctrl+C and their output still reach the
terminal. Only a process without a console window (the desktop app)
starts them with CREATE_NO_WINDOW. The choice is logged at startup and
covered by a Windows-only test that runs itself as a child with and
without a console.
* ci(desktop): link to the upstream FFmpeg source instead of uploading it to releases
Drop the ffmpeg-source job that packed about 2 GB of FFmpeg and dependency
source into every release. The installer's ffmpeg/README.txt and the release
notes now point to the FFmpeg commit and the BtbN/FFmpeg-Builds tree of the
bundled build (full commit hashes, resolved at build time), whose scripts.d/
pins the version and source URL of every library. The Docker image uses the
same BtbN tag, so the same links cover it.
* feat(cli): let serve run on a host-bound listener with a host-owned shutdown
ServeOptions gains two optional fields for embedding hosts:
- listener: serve on an already-bound socket, so the host knows the
actual port (e.g. after falling back to port 0);
- shutdown: stop when the given future completes instead of installing
Ctrl+C / SIGTERM handlers, which would otherwise take those signals
away from the host process.
The CLI path is unchanged apart from binding the port before opening the
database, so a busy port now fails before data/ is touched.
Ref #1721
* feat(desktop): run biliup server in-process, drop PyInstaller sidecar
The Tauri app now depends on biliup-cli as a library and runs
entry::serve on its own tokio runtime instead of spawning
binaries/biliup.exe:
- listens on 127.0.0.1:19159, or a free port when that one is taken;
the window shows a bundled startup page and switches to the Web UI
once GET / answers 200 (startup failures are shown on that page);
- data/, ds_update.log, cookies.json and recordings live in
app_data_dir(); on first start, data/ left in the install directory
by the old sidecar is copied over and the original is kept;
- closing the window stops the server and waits for it (up to 10 s);
no expect() on exit;
- a second launch focuses the running window instead of starting a
second server on the same data.
Removed: the greet command, the Vite + React scaffold under
tauri-app/src, the shell:allow-execute capability, tauri-plugin-shell,
tauri-plugin-opener, GBK decoding of sidecar stdout and the bundled
binaries/ resource. The Web UI itself is unchanged.
Ref #1721
* ci(desktop): build the Tauri app directly, remove biliup.spec
The Windows desktop workflow no longer builds a wheel, installs it and
runs PyInstaller: it builds the Web UI (embedded into biliup-cli by
rust-embed) and then the Tauri app with tauri-action. Renamed
pyinstaller-publish.yml to desktop-publish.yml.
On release it uploads to the release as before; a manual
workflow_dispatch run only builds and uploads the installers as an
artifact, without touching any release.
The Dockerfile used biliup.spec to tell whether it was built from a
checkout (otherwise it clones master); it now checks pyproject.toml.
Ref #1721
* ci(desktop): smoke-test the installer on manual runs
Only on workflow_dispatch: pretend an old install left data/ in the
install directory, install the NSIS package silently, start the app and
require GET http://127.0.0.1:19159/ to return 200, the database to be
created in the app data directory, the old data/ to be copied there and
kept in place, and the server to stop when the window is closed.
Ref #1721
* feat(desktop): keep data next to the exe unless it is on the system drive
On Windows the data directory is the install directory again, as with the
old sidecar, when the executable is not on %SystemDrive%. When it is, the
first start asks the user (tauri-plugin-dialog): the app data directory
(default), the install directory, or another folder. The choice is saved
as {"data_dir": ...} in desktop.json under the app config directory and
not asked again; a saved directory that cannot be used is reported on the
startup page instead of silently creating a new one. Closing the dialog
quits.
Other platforms have no drive letters and install into read-only
locations, so they keep using the app data directory (or data_dir from
desktop.json) without asking.
When the chosen directory has no data/ yet, data/ left by an older
version is copied there and kept in place. Besides the install directory,
Windows also looks where the old bbup-app installers put the app (their
registry keys and %LOCALAPPDATA%\bbup-app), since a new product name
installs into a new directory.
The data directory is resolved on a separate thread because the dialog
blocks. The migration helpers move to data_dir.rs unchanged.
Ref #1721
* chore(desktop): replace the template names with biliup
- productName bbup-app -> biliup (installers biliup_<version>_x64-setup.exe
and biliup_<version>_x64_en-US.msi, NSIS default install directory
%LOCALAPPDATA%\biliup, Start menu entry biliup)
- identifier com.tauri-app.app -> com.biliup.desktop (app data directory
%APPDATA%\com.biliup.desktop, which is the default data directory
offered on the system drive)
- Cargo package tauri-app -> biliup-desktop (executable
biliup-desktop.exe, so it is not confused with the biliup CLI), lib
tauri_app_lib -> biliup_desktop_lib, drop the template authors
- npm package name, workflow artifact name and the README download link
Tauri derives the NSIS uninstall/registry keys and the MSI upgrade code
from the product name, so the new installers do not replace an installed
bbup-app; both show up in Apps & features. The app copies data/ from the
old bbup-app install directory on first start (see the previous commit),
after which bbup-app can be uninstalled; its uninstaller does not delete
data/.
Ref #1721
* ci(desktop): bump the workflow actions to their latest majors
actions/checkout v4 -> v7, actions/setup-node v4 -> v7,
actions/upload-artifact v4 -> v7 (Node 24 runtimes; the other breaking
changes are about fork checkouts in pull_request_target, caching driven by
a packageManager field and single-file uploads, none of which this
workflow uses). Swatinem/rust-cache@v2 and dtolnay/rust-toolchain@stable
already track the latest release.
tauri-apps/tauri-action v0 -> v1. v1 updates the name and body of an
existing published release it finds by tagName with releaseName and
releaseBody, which are empty here and would wipe the release notes, so
the release job now passes releaseId from the release event instead;
with releaseId the action uploads to that release without looking it up.
Ref #1721
* ci(desktop): smoke-test the new data directory rules on manual runs
Replaces the manual-run smoke step from the earlier smoke-test commit and
can be dropped together with it. On the Windows runner:
- install the v1.2.8 bbup-app NSIS package into C:\old-bbup-app and leave
a data/ there;
- install biliup into its default directory on the system drive (with an
old data/ in it); both products stay installed side by side;
- first start shows the data directory dialog and does not start the
server; UI Automation clicks the default button (the dialog text and a
screenshot are kept as evidence), then GET / must return 200, the
database and the copied data/ must be in %APPDATA%\com.biliup.desktop,
the original kept and the choice saved in desktop.json;
- a second start must not ask again; a preset desktop.json pointing at
another directory stands in for "choose another folder";
- reinstall on D: (or a subst drive when the runner has none): no dialog,
database and log in the install directory, data/ copied from the old
bbup-app directory found through its registry key;
- uninstalling bbup-app keeps its data/.
Ref #1721
* Revert "ci(desktop): smoke-test the new data directory rules on manual runs"
This reverts commit 860c45e89f8e5dab11468c83db7c623b7304e4dd.
* Revert "ci(desktop): smoke-test the installer on manual runs"
This reverts commit b3831f014a2b37f81b71c96e8229a6254bee1b9c.
* fix(stream-gears): write the cached GOP on disconnect, flush before the segment hook
The httpflv loop kept the current GOP in `flv_tags_cache` and only wrote it
when the next keyframe arrived. On EOF, a 30 s read timeout or a parse error
the function returned and the cache was dropped, so every disconnect,
reconnect and stream end lost the last 1-10 s. The loop now runs in an inner
function; whatever way it exits, the complete tags in the cache are written
before the original result (including the error) is returned.
`FlvFile` / `TsFile` renamed the `.part` file and ran the segment hook before
the old `BufWriter` was flushed, so the hook (FileValidator, upload) could see
a truncated file, and a failed final flush (disk full) was swallowed. Both now
go through `finish()`: flush and check the error, then rename, then run the
hook. `create_new` returns the flush error; `Drop` logs it with `error!`.
Tests: the rolling-segment test now expects 120 keyframes on disk (was 119)
and checks the size seen by the hook equals the final size; new cases for a
stream that ends cleanly, a read that times out and a parse error; a TS hook
size test; a failed-flush test.
Ref #1726
* feat(recorder): default to mesio when no downloader is configured
Rooms without a `downloader` setting followed the plugin hint, which is
stream-gears for almost every platform. They now use mesio: its FLV output
carries `onMetaData.keyframes` and every segment starts at timestamp 0, so
players can seek; it also records HEVC / Enhanced-FLV and HLS fMP4, which
stream-gears cannot. Only the hint mapping in `core/live.rs` changes; an
explicit `stream-gears` keeps using stream-gears, and plugins that hint
streamlink / yt-dlp are unaffected.
`DownloaderRuntime::from_type` no longer has a catch-all arm. `ffmpeg-external`
and `ffmpeg-internal` used to fall into it and silently run stream-gears; both
now run ffmpeg (external segmenting, same as `ffmpeg`). The arm for
streamlink / yt-dlp / ytarchive, which need the live stream and are built in
`downloader_runtime`, is listed explicitly and logs a warning.
WebUI, README, sample configs and CHANGELOG now say mesio is the default and
list the behaviour differences: the container follows the source (HLS fMP4 is
saved as .mp4), files are written under their final name without `.part`, and
Douyu Wangsu reconnects leave a 2.6-5.8 s overlap between segments.
Ref #1726
* feat(mesio): pass segment duration and size to the segment callback
mesio's segment-complete callback reports the duration and byte size of each
closed file, but biliup only logged them and built `SegmentInfo` with the path
and index. `SegmentInfo` gains `duration_secs` / `size_bytes` (None for the
other downloaders) and mesio fills them in. The end-to-end mesio test checks
the reported size equals the file size on disk.
Ref #1726
* fix(recorder): parse segment_time the same way in every downloader
`segment_time` was read three different ways. stream-gears indexed
`parts[1]` / `parts[2]` directly, so "3600" or "01:00" panicked the download
task (and an unparsable hour silently became 1); mesio returned 0 for anything
that was not HH:MM:SS and silently recorded without time splits; ffmpeg got
the raw string and accepted both forms.
`parse_segment_time` follows ffmpeg's duration syntax (`[HH:]MM:SS[.frac]`
or plain seconds; minutes and seconds below 60 when colons are used), so all
three downloaders now agree. stream-gears and mesio share
`DownloadConfig::segment_time_limit`: 0 means no time split for both (mesio
already did this; stream-gears used to split at every keyframe), and a value
that still cannot be parsed logs a warning and records without time splits
instead of panicking or being ignored silently. The time-range clamp uses the
same parser, so "3600" is shortened near the window end like "01:00:00".
Ref #1726
* feat(webui): enter file_size as a number with an MB/GB unit
The config still stores bytes. The control converts with 1024 (the same
sizes Windows Explorer shows), keeps an untouched value byte-exact even
when it is not a round MB/GB, and re-picks the unit when the form value
is replaced from outside.
In the streamer override, clearing a value that was already set now
saves null (no size split for that streamer); before, Semi dropped the
emptied key and the old value stayed. Clearing something typed in the
same session still falls back to inheriting the global setting.
* fix(webui): keep the file_size unit beside the number in narrow modals
* feat(uploader): public season API handle and BV lookup for archive view
- `SeasonApi` becomes public: `BiliBili::season_api()` returns it, and a
`#[doc(hidden)]` `SeasonApi::with_hosts` points it at other hosts so
callers outside the crate (the CLI tests) can use a local fake server.
The `BiliBili` struct itself is unchanged.
- `archive_view_by_vid(&Vid)` queries the public view endpoint with either
`aid` or `bvid`, so a BV number does not need a separate lookup.
Ref #1721
* feat(cli): biliup season subcommands
`biliup season list | sections <season_id> | add <section_id> --vid … |
remove <episode_id> | sort <section_id> [episode_id…] [--reverse]`, all with
`--json`. `add` fetches cid and title through the public view endpoint.
`sort` puts the listed episodes first and keeps the rest in their current
order, since B站 requires every episode of the section in the request.
The logic lives in `season_cli.rs`; dispatch only gains one match arm in
`main.rs` and in the copy in `stream-gears/src/server.rs`.
Ref #1721
* docs: list season and user subcommands in README and SKILL.md
Ref #1721
* refactor(cli): single entry for native binary and main_loop
Move argument parsing, logging setup and command dispatch from
biliup-cli's main.rs into `biliup_cli::entry`, and make
stream_gears.main_loop call the same entry instead of its own copy.
- Both entries default to `server` when no subcommand is given (or the
legacy `start`), expand `~` in paths, and log to `ds_update.log`
(server only, no rotation) instead of `download.log` on the wheel.
- The log filter honours `RUST_LOG`; an explicit `--rust-log` overrides it.
- main_loop no longer calls process::exit on clap errors: `--help` and
`--version` print and return, other parse errors raise SystemExit with
clap's exit code. Logging uses try_init, so a second call in the same
process does not panic.
- Export `entry::serve(ServeOptions)` so a host can run the Web server on
its own tokio runtime.
- Document config_bindings as deprecated (behaviour unchanged).
Ref #1721
* test(stream-gears): cover the shared entry from the installed wheel
Exit codes of --help/--version/--bad-flag, main_loop called twice in one
process, and biliup server writing ds_update.log with RUST_LOG and
--rust-log precedence.
Ref #1721
* test(stream-gears): update smoke test comment now that --help no longer exits
* build(stream-gears): drop clap, unused after the shared entry
* ci(docker): start the built image and check GET / returns 200
The test job only built the image. Also run it with the default CMD and
require the Web UI to answer and ds_update.log to be written.
Ref #1721
* build(stream-gears): enable pyo3 extension-module via default crate feature
Plain cargo build/test of stream-gears no longer links libpython, so it
works without Python dev files. build.rs adds the macOS cdylib link args
that maturin would otherwise pass. Wheel builds are unaffected: maturin
already enabled pyo3/extension-module through pyproject.toml.
Ref #1721
* build(stream-gears): drop 13 unused dependencies and biliup-cli python-bridge feature
bytes, ormlite, rust-embed, mime_guess, tower-http, sqlx, serde,
thiserror, futures, async-channel, url, chrono and fancy-regex have no
references under crates/stream-gears/src. The python-bridge feature of
biliup-cli is declared but never used by any cfg or dependent crate.
Ref #1721
* ci(test): run cargo test for the whole workspace including stream-gears
Ref #1721
* feat(web): multi-user accounts with fixed admin/operator/viewer roles
- migration 6 adds web_users and copies the existing single admin with
the same id and password hash, so upgrades need no action and existing
sessions stay logged in
- table-driven permission middleware on every business route (default
deny), 403 for missing permission, 401 when logged out
- hooks/override and secrets redacted for non-admins; operators may only
pick registered bilibili accounts for templates
- /v1/web-users management API, /v1/me and self password change
- login rate limit: 5 consecutive failures lock the username for 30s
- revoked sessions cut long-lived websocket/stream responses
Refs #1717
* feat(cli): add `biliup user list` and `biliup user reset-password`
Refs #1717
* feat(web-ui): user management page, account menu and editable login name
- /users page (super admin only): create users, change role, reset
password, disable, force logout, delete; table on desktop, cards on
mobile, with loading/empty/error states
- sidebar filters menu entries by the permissions from /v1/me and shows
a no-access state for pages the role cannot open
- account menu in the sidebar footer with change password and logout,
hidden when --auth is off
- 403 responses show a warning toast instead of redirecting
- login page username is editable (default biliup)
Refs #1717
* feat(web-ui): hide actions the current role cannot perform
Streamer add/edit/delete/pause, hook and override editing, template and
upload actions, bilibili account management and config saving are only
rendered for roles that have the matching permission. Account and type
tree requests are skipped for roles that would get 403.
Refs #1717
* fix(web-ui): wait for permissions on restricted pages, surface pause errors
- pages that need a permission not every role has render a spinner until
/v1/me returns, so roles without it never fire requests doomed to 403
- user management is hidden when --auth is off, with an explanation if
the page is opened directly
- the pause button goes through the shared response handler, so a 403
shows the permission toast instead of failing silently
- permission errors sent as plain text keep their message
Refs #1717
* fix(web-ui): keep user actions reachable on tablet and narrow desktop
Refs #1717
* fix(py): handle the user subcommand in the stream-gears entry point
Refs #1717
Add an in-process mesio downloader using the rust-srec mesio-engine. Introduces crates/biliup-cli/src/server/core/downloader/mesio.rs implementing FLV/HLS pull, timestamp fixes, keyframe indexing, and size/duration-based segmentation with pipeline_common, flv_fix and hls_fix. Wire Mesio into downloader enum/runtime and add UI options and public config docs. Update crates/biliup-cli/Cargo.toml and workspace Cargo.toml/Cargo.lock to include new dependencies and bump version to 1.2.5. Include tests for protocol selection, pipeline behavior and an e2e FLV download. Small helper changes: config enum docs and segment handling.
Add optional tid_v2 across Studio, config/DB, CLI, Web UI, stream-gears,
and Python uploaders so new Bilibili partition IDs can be set alongside
legacy tid. Fixes#1662; related to #1562.
Prefix already-resolved progressive HTTP(S) media URLs (e.g. Huya/Douyu
.flv CDN links) with httpstream:// so Streamlink's http plugin matches
them, while leaving HLS/DASH and plugin webpage URLs unchanged.
When a download returns Error (or Err) while the room is still Live,
keep exponential retry backoff instead of resetting retry_count to 0,
which previously caused "Retrying download in 0ns..." spam. Immediate
retry remains only for successful segment completion / clean stream end.
Co-authored-by: ransxd <ransxd@users.noreply.github.com>
This change updates the server default bind to 127.0.0.1 for safer local-only access while documenting how to expose the Web UI via --bind 0.0.0.0 --auth for remote or Docker use. It also removes the previous restriction that blocked first-time admin bootstrap on non-loopback addresses, allowing authenticated remote initialization and bumping the workspace version to 1.2.4.
This change adds a recording policy layer that enforces time-window eligibility and excluded title keywords before probing and before recording. It updates the streamer API to surface reasons like OutOfSchedule and TitleExcluded, clamps ffmpeg/segment durations to the configured window, and exposes the last rejection reason from the worker state for the frontend.
启动 Web 服务时自动登记 CLI 指定的 B 站凭据，并为账号登记、刷新和删除统一补上并发去重、原子写入及引用检查。
同时收紧默认监听、管理员初始化、认证路由、静态文件和日志 WebSocket，恢复 server 的 ds_update.log，并避免凭据及上游响应进入日志。
新增按账号查询远程稿件的 API 与 Web 页面，补齐严格分页和响应校验。
关联 biliup/biliup#1624
关联 biliup/biliup#1658
关联 biliup/biliup#724
Introduce batch live-check infrastructure and implementations. Add BatchCheckRequest and LivePlugin::supports_batch_check/batch_check API; Monitor uses BatchLiveCache and batch_verdict to prefilter rooms (platform-level checks + caching and fast rotation). RoomsActor gains PlatformUrls. Implement batch_check for Picarto (explore pagination) and Twitch (GQL batching with auth retry). Other fixes: Douyu auth-failure detection and key invalidation + retry, Inke optional response handling, streamlink runtime tweaks for yt-dlp/Twitch ffmpeg, small test additions and import refactors.
Set CFLAGS_powerpc64le_unknown_linux_gnu=-DAT_HWCAP2=26 in CI to work around aws-lc-sys (via rustls) calling getauxval(AT_HWCAP2) on manylinux2014 where glibc 2.17 headers lack AT_HWCAP2; glibc still provides getauxval so the value is safe and only used for ppc64le cc invocations. Also enable the tokio "io-std" feature for the biliup-cli crate to opt into the io-std I/O implementation.
Bump dependencies (Cargo.toml + Cargo.lock) to newer crate versions and adapt code to API changes. Refactor server-side upload/download logic, the upload_streamer model, and related API endpoints to improve streaming handling, buffer/error propagation and compatibility with updated crates (e.g. tokio-tungstenite/futures/etc.). These changes align the codebase with the upgraded dependency set and fix streaming/endpoint issues introduced by those upgrades.
When the upload template's reprint source (copyright_source) is left
empty, the web form submits an empty string rather than null, so it is
stored as Some(""). build_studio used `unwrap_or_else`, which only fires
on None, so the submitted source became an empty string instead of the
live room URL — contradicting the UI hint
"如不填写转载来源默认为直播间地址".
Treat None and whitespace-only values as "unset" and fall back to the
live room URL. Extract the resolution into a pure `resolve_source`
helper with unit tests covering none / empty / whitespace / real value.
Make file_size and segment_time optional and normalize/validate their values so segmentation can be disabled. Frontend: send nulls for empty fields and allow empty segment_time via validation; add showClear to inputs. Downloader: extend SegmentInfo with optional danmaku_file_path and propagate it from various downloaders; DanmakuClient::rolling now returns a bool to signal whether a danmaku file was produced. Uploader: add segment_paths helper to include danmaku files in upload/postprocessing, and preserve processed paths; add unit tests. HookStep: stream child process stdout/stderr as raw bytes to terminal and append to a log file (tee), and simplify run/move logic to use explicit path lists. Cargo.toml: bump reqwest to 0.13.4. Tests and minor adjustments added throughout to support these changes.
Migrate the server-side live downloading to the new biliup::downloader::live API and simplify the actor model. Key changes:
- Introduce a new server/core/live module that builds LiveRequest/LiveOptions/LiveCredentials, maps runtimes, and creates danmaku clients.
- Replace legacy plugin/DownloadBase actor flow with LivePlugin-based workflows and a start_download_workflow entrypoint; monitor now acquires a Semaphore permit before starting a download to enforce concurrency.
- Remove the old plugin module and per-platform plugin plumbing; many platform implementations were moved/renamed under crates/biliup/src/downloader/live.
- Adjust DownloadManager, Monitor, and common download/upload code to use LivePlugin, LiveStream and related types; pass uploader channel and download slot semaphore into Monitor.
- Add download_stream helper to support direct FLV/HLS downloads via StatelessClient, and extend Streamlink downloader to support Niconico credentials/options.
- Update usage of context/streamer info and danmaku initialization to the new LiveStream-based model.
- Bump workspace version to 1.2.0 (Cargo.toml) and related package versions.
This refactor centralizes live logic on the shared biliup downloader live API, simplifies concurrency control, and removes the previous actor-based plugin indirection.
Remove the legacy Python biliup/Danmaku package and related douyin utilities, and adjust PyInstaller spec to stop bundling those resources. Introduce and expand Rust-based danmaku and plugin implementations (crates/danmaku, crates/biliup-cli server/core/plugin/*, and stream-gears changes) and update various server/downloader modules to integrate them. Also add a TOML dependency to Cargo.toml to support the new Rust changes.
* feat(hook): built-in remux:mp4 hook + activate segment_processor stage
B站 transcoder rejects raw .ts uploads with PTS jumps ("您的视频未能成功
转码 ... 该视频内部存在时间戳跳变"), which is common on HLS reconnects
mid-segment. The fix is to feed it an mp4 with monotonic PTS — a plain
ffmpeg copy-remux is enough.
Add a built-in `HookStep::Remux { remux: String }` variant that runs
`ffmpeg -fflags +genpts+igndts -c copy -bsf:a aac_adtstoasc
-movflags +faststart -avoid_negative_ts make_zero` on .ts/.m2ts inputs.
Lossless and idempotent (reuses existing .mp4 sibling). Currently
target="mp4" only.
To make this useful, segment_processor (defined in config but never
previously invoked) is now executed in pipeline_upload_videos BEFORE
each upload_single_file. That's the right stage for ts→mp4 — must
run before pre_upload, not after.
Path-mutating processor pipeline:
- new execute_paths(&mut Vec<PathBuf>) API on HookStep
- new process_video_paths helper
- Remux replaces .ts entries with the new .mp4 entries so downstream
stages (upload, postprocessor) see the right paths.
The legacy execute(&[&Path]) path stays for postprocessor-style steps
that don't mutate the working set.
Config form:
segment_processor:
- {remux: "mp4"}
Tests cover YAML/JSON deserialization of all 4 HookStep variants
and Remux skipping non-.ts inputs.
* fix(upload): 单段 segment_processor / upload 失败不再拖死整批
之前 pipeline_upload_videos 里两个 ? 把单段失败直接抛到
process_with_upload，让后续 submit + execute_postprocessor（含 rm）
全部跳过。在磁盘紧的场景下，ffmpeg remux 写 mp4 header 时遇到
"No space left on device" → segment_processor Err → 整个直播会话已
成功上传的段也不会触发本地 rm → 文件堆积 → 磁盘更紧 → 下一段又失败，
正反馈把磁盘吃完。
把两处 ? 改成 log + continue：
- segment_processor 失败：跳过当前段，下个事件继续
- upload_single_file 失败：跳过，已上传的段照常进入 uploaded.{videos,paths}
直播会话结束（channel close）时，submit_to_bilibili 仍然能用收集到
的 videos 提交，execute_postprocessor 仍能 rm 已成功的本地文件。
修复对应残留 1.6 TB 历史文件的根因（35 次/5 天 "Process segment
event failed: ffmpeg remux failed ... No space left on device"）。
* test(hook): remux 在 ffmpeg 失败时返回 Err（供 pipeline 容错前置条件）
复现 pipeline_upload_videos 容错路径的触发条件：segment_processor
(Remux step) 在 ffmpeg 退非 0 时返回 Err。生产事故的具体表现就是磁盘
满让 ffmpeg 写 mp4 header 失败，调用方必须能从该 Err 优雅恢复
（log + continue），否则整批已上传的段都拿不到 postprocessor 的 rm。
完整 pipeline_upload_videos 的端到端测试要 mock UploadContext
(BiliBili client + 登录态)，这是后续重构 seam 的方向。
Unify and simplify how hook Run steps execute external commands: execute_command now accepts raw stdin bytes (src: &[u8]) and runs the full command string via the platform shell (cmd /C or sh -c), writing src to the child's stdin, streaming stdout/stderr lines to logs, and checking exit status. execute_with for the Run case now builds a newline-separated paths string (when called with file paths) and delegates to execute_command. Removed manual arg parsing and duplicated process-handling logic. Also bumped workspace/package version to 1.1.29 (Cargo.toml) and updated lockfile accordingly.
Introduced a new cover_downloader module in the downloader core. Updated several dependencies in Cargo.toml and Cargo.lock, including major upgrades to image processing, HTTP, and async libraries. Adjusted related code in util, download, plugin, and uploader modules to support new features and dependency changes.
ForgQi