* fix(build): restore editable installs after stream_gears packaging change
Editable installs only put python-source (crates/stream-gears) on sys.path and skip include, so the root biliup/ package was unimportable after maturin develop / pip install -e . Add crates/stream-gears/biliup/__init__.py, which loads the root package under the same name. Wheels are unchanged: maturin only walks stream_gears/ and the include globs, which match the root biliup/ first.
* test(stream-gears): cover editable installs
test_wheel_ships_stubs only checks RECORD for regular installs, since an editable install's RECORD lists the .pth. A new test puts only python-source on PYTHONPATH and checks that biliup and biliup.__main__ resolve to the root package.
* build: move biliup/ and pyproject.toml into crates/stream-gears
Editable installs (`maturin develop`, `pip install -e`) only put
`python-source` on sys.path and skip `include`, so with `biliup/` at the
repository root they could not import it. With `pyproject.toml`,
`biliup/` and `stream_gears/` in the same directory, `biliup` goes in
through `python-packages`, which editable installs honour.
- `biliup/` is moved unchanged; the loader added earlier in this PR is
removed because that path is now the real package.
- `readme` and `license` point at the repository root, so the wheel's
METADATA and `licenses/LICENSE` stay the same.
- CI, Termux and the Dockerfile build with
`-m crates/stream-gears/Cargo.toml`.
Ref #1732
* docs: point source install commands at crates/stream-gears
* fix(stream-gears): write the cached GOP on disconnect, flush before the segment hook
The httpflv loop kept the current GOP in `flv_tags_cache` and only wrote it
when the next keyframe arrived. On EOF, a 30 s read timeout or a parse error
the function returned and the cache was dropped, so every disconnect,
reconnect and stream end lost the last 1-10 s. The loop now runs in an inner
function; whatever way it exits, the complete tags in the cache are written
before the original result (including the error) is returned.
`FlvFile` / `TsFile` renamed the `.part` file and ran the segment hook before
the old `BufWriter` was flushed, so the hook (FileValidator, upload) could see
a truncated file, and a failed final flush (disk full) was swallowed. Both now
go through `finish()`: flush and check the error, then rename, then run the
hook. `create_new` returns the flush error; `Drop` logs it with `error!`.
Tests: the rolling-segment test now expects 120 keyframes on disk (was 119)
and checks the size seen by the hook equals the final size; new cases for a
stream that ends cleanly, a read that times out and a parse error; a TS hook
size test; a failed-flush test.
Ref #1726
* feat(recorder): default to mesio when no downloader is configured
Rooms without a `downloader` setting followed the plugin hint, which is
stream-gears for almost every platform. They now use mesio: its FLV output
carries `onMetaData.keyframes` and every segment starts at timestamp 0, so
players can seek; it also records HEVC / Enhanced-FLV and HLS fMP4, which
stream-gears cannot. Only the hint mapping in `core/live.rs` changes; an
explicit `stream-gears` keeps using stream-gears, and plugins that hint
streamlink / yt-dlp are unaffected.
`DownloaderRuntime::from_type` no longer has a catch-all arm. `ffmpeg-external`
and `ffmpeg-internal` used to fall into it and silently run stream-gears; both
now run ffmpeg (external segmenting, same as `ffmpeg`). The arm for
streamlink / yt-dlp / ytarchive, which need the live stream and are built in
`downloader_runtime`, is listed explicitly and logs a warning.
WebUI, README, sample configs and CHANGELOG now say mesio is the default and
list the behaviour differences: the container follows the source (HLS fMP4 is
saved as .mp4), files are written under their final name without `.part`, and
Douyu Wangsu reconnects leave a 2.6-5.8 s overlap between segments.
Ref #1726
* feat(mesio): pass segment duration and size to the segment callback
mesio's segment-complete callback reports the duration and byte size of each
closed file, but biliup only logged them and built `SegmentInfo` with the path
and index. `SegmentInfo` gains `duration_secs` / `size_bytes` (None for the
other downloaders) and mesio fills them in. The end-to-end mesio test checks
the reported size equals the file size on disk.
Ref #1726
* fix(recorder): parse segment_time the same way in every downloader
`segment_time` was read three different ways. stream-gears indexed
`parts[1]` / `parts[2]` directly, so "3600" or "01:00" panicked the download
task (and an unparsable hour silently became 1); mesio returned 0 for anything
that was not HH:MM:SS and silently recorded without time splits; ffmpeg got
the raw string and accepted both forms.
`parse_segment_time` follows ffmpeg's duration syntax (`[HH:]MM:SS[.frac]`
or plain seconds; minutes and seconds below 60 when colons are used), so all
three downloaders now agree. stream-gears and mesio share
`DownloadConfig::segment_time_limit`: 0 means no time split for both (mesio
already did this; stream-gears used to split at every keyframe), and a value
that still cannot be parsed logs a warning and records without time splits
instead of panicking or being ignored silently. The time-range clamp uses the
same parser, so "3600" is shortened near the window end like "01:00:00".
Ref #1726
* feat(webui): enter file_size as a number with an MB/GB unit
The config still stores bytes. The control converts with 1024 (the same
sizes Windows Explorer shows), keeps an untouched value byte-exact even
when it is not a round MB/GB, and re-picks the unit when the form value
is replaced from outside.
In the streamer override, clearing a value that was already set now
saves null (no size split for that streamer); before, Semi dropped the
emptied key and the old value stayed. Clearing something typed in the
same session still falls back to inheriting the global setting.
* fix(webui): keep the file_size unit beside the number in narrow modals
Add optional tid_v2 across Studio, config/DB, CLI, Web UI, stream-gears,
and Python uploaders so new Bilibili partition IDs can be set alongside
legacy tid. Fixes#1662; related to #1562.
Migrate the server-side live downloading to the new biliup::downloader::live API and simplify the actor model. Key changes:
- Introduce a new server/core/live module that builds LiveRequest/LiveOptions/LiveCredentials, maps runtimes, and creates danmaku clients.
- Replace legacy plugin/DownloadBase actor flow with LivePlugin-based workflows and a start_download_workflow entrypoint; monitor now acquires a Semaphore permit before starting a download to enforce concurrency.
- Remove the old plugin module and per-platform plugin plumbing; many platform implementations were moved/renamed under crates/biliup/src/downloader/live.
- Adjust DownloadManager, Monitor, and common download/upload code to use LivePlugin, LiveStream and related types; pass uploader channel and download slot semaphore into Monitor.
- Add download_stream helper to support direct FLV/HLS downloads via StatelessClient, and extend Streamlink downloader to support Niconico credentials/options.
- Update usage of context/streamer info and danmaku initialization to the new LiveStream-based model.
- Bump workspace version to 1.2.0 (Cargo.toml) and related package versions.
This refactor centralizes live logic on the shared biliup downloader live API, simplifies concurrency control, and removes the previous actor-based plugin indirection.
Refactored the DownloadTask::execute method to return DownloadStatus and adjusted its visibility. Enhanced DActor message handling to update stream info after download execution. Updated README and introduction guide with improved instructions for background running and systemd service configuration.
Noah