Commit Graph
1006 Commits
Author SHA1 Message Date
Noah 40234133e3 fix(webui): 保存配置覆写时保留投稿模板关联 (#1685)
覆写弹窗的 entity 白名单漏了 upload_streamers_id,PUT 整行覆盖后外键变 NULL,后续录像不再投稿并走默认 rm。
2026-09-13 15:40:03 +00:00
Noah 9445db636b fix(danmaku): 虎牙弹幕从用户结构体读取昵称和正文 (#1689)
MessageNotice 的 tag 2 是 lSid 不是用户名。旧解析把 lSid 当字符串读,lSid 非 0 时连正文都对不齐,弹幕几乎全部丢弃。
2026-09-13 15:36:15 +00:00
Noah 8437b028c3 fix(db): 迁移 2 将空 tags 写成 [] 而不是 NULL (#1684)
uploadstreamers.tags 是 JSON NOT NULL。SET tags = null 会让从 Python 升级且 tags 为空字符串的实例卡在迁移 2。
2026-09-13 15:33:12 +00:00
Noah 1a56242696 fix(log): 下载器命令行日志脱敏 Cookie 和 OAuth (#1688)
* fix(log): 下载器命令行日志脱敏 Cookie 和 OAuth

ffmpeg/streamlink/yt-dlp/ytarchive 会把 Cookie 头和 auth-token 打进 ds_update.log,该文件可经 Web 日志页与 /static 下载,排障贴日志即泄露登录态。

* fix(log): 头部值整段脱敏并合并测试到已有 tests 模块

Cookie/Authorization 的值吃到下一个引号,OAuth token 不再漏出;新增 --xxx-password 形式 argv 的脱敏;sid 不再当密钥以免误伤斗鱼直链。util.rs 已有 mod tests,第二个会 E0428。
2026-09-13 15:31:43 +00:00
Noahandransxd 86bcaffd2a fix(sync): AUTO 测速超时与 file_size 覆写占位 null (#1693)
- AUTO 线路探测为列表请求与单线路测速增加 15s 硬超时,失败只跳过该线路
- POST 探测体按服务端 probe.post(默认 0.1MB)发送,避免固定 10MB 拖慢选线
- ConfigPatch 序列化跳过未设置的 file_size,避免落库/回传把全局值清成 None
- 迁移清理历史 override 中的 file_size: null 占位
- 同步保留边录边传管线健壮性修复(取消保留分段、错误提示、分段顺序等)

Co-authored-by: ransxd <ransxd@users.noreply.github.com>
2026-09-13 15:29:12 +00:00
Noah 788682398a fix(docker): FFmpeg 固定到 BtbN 月末 autobuild (#1691)
autobuild-2026-08-29 已被上游删除,Docker 构建 404。BtbN 只长期保留每月最后一天的构建,改为固定 autobuild-2026-08-31-13-27 并更新 SHA-256。
2026-09-12 18:39:59 +00:00
Noah 906e0f6fdb docs: 依据代码实现重写 README,迁移弃用域名并修正失效信息 (#1681)
* docs: 依据代码实现重写 README,修正失效信息并补充平台与开发说明

保持原有结构、标题与徽章排版不变,按当前代码逐项校对内容。

修正与实现不符之处:
- 登录信息文件名写作 cookie.json,实际 CLI 默认值为 cookies.json
  (crates/biliup-cli/src/cli.rs 及其单测均为 cookies.json)
- Windows 下载链接硬编码 bbup-app_0.1.0_x64,随发版即失效,改为
  指向 Release 页
- Python 徽章链接 http://www.python.org/download 返回 404,改为
  https://www.python.org/downloads/
- Node.js 要求由 ≥ 18 修正为 ≥ 18.17(Next.js 14 的实际下限)

新增内容:
- 支持平台表:依据 builtin_plugins() 的 19 个平台适配器与通用兜底
  适配器,站点域名取自各适配器正则;弹幕列依据 danmaku
  create_platform() 支持的 7 个平台
- download 子命令用法示例与分段参数说明
- 外部依赖说明:默认 stream-gears 为纯 Rust 实现无需外部程序,
  ffmpeg 下载器与后处理需 ffmpeg,YouTube/niconico 及通用适配器
  需 yt-dlp 或 streamlink
- 目录结构表,以及 cargo 构建前必须先 npm run build 的说明
  (biliup-cli 经 rust-embed 内嵌 out/,缺失会导致编译失败)
- 测试命令与 Docker /opt 持久化、首次访问尽快初始化管理员的提示

校验:CLI 帮助文本与 target/debug/biliup --help、server --help 实际
输出逐字一致;远程访问拒绝启动的报错取自实际运行输出;mermaid
架构图经 mermaid-cli 渲染通过;README 全部外链已联网检查。

* docs: 迁移已弃用的 biliup.rs 文档域名,移除失效的 winget 安装方式

README:
- 顶部 logo 原引用 docs.biliup.rs/home.png,该域名已弃用且无法访问,
  改为仓库内 public/logo.png 的 raw 链接。README 同时用作 PyPI 长描述
  (pyproject.toml readme = "README.md"),故使用绝对地址而非相对路径
- 「使用文档」链接改为新域名
  https://biliup.github.io/biliup/docs/guide/introduction/

SKILL.md:
- 删除 Windows winget 安装段落。winget 上不存在名为 biliup 的包,
  该命令返回 No package found matching input criteria;现有的
  ForgQi.biliup-app(0.6.7)、ForgQi.biliup-rs(0.2.4) 均已停止更新,
  远落后于当前 1.2.4。Windows 用户仍可通过既有的 GitHub Releases
  段落(含 PowerShell 脚本)安装

校验:README 全部 29 个链接联网复查,除 GitHub 对未登录请求的
/stargazers 统一返回 404(rust-lang/rust 同样如此,属反爬行为)外
均为 200;新 logo 地址返回 200、image/png、65525 字节。
2026-09-01 13:49:39 +00:00
Noah c4dbf81c5d fix(danmaku): B站数据包解析不再对畸形输入 panic,解码 panic 不再终结录制 (#1678)
decode_packet 只校验 packet_len 是否越界,未校验其不小于 16 字
节包头:远端发来 packet_len < 16 的畸形帧时,body 反向切片直接
panic;panic 发生在 tokio::spawn 的录制任务里,run() 的重连逻辑
只处理 Err,任务被杀死后没有任何重启机制,弹幕录制静默永久停
止,XML 文件也不会 finalize。

修复:
- packet_len < 16 视为不可信帧,丢弃剩余数据并保留已解析的包,
  同时消除 packet_len=0 时 offset 不前进的死循环隐患
- 压缩包递归解码增加深度上限(真实协议仅一层嵌套),恶意多层
  嵌套不再导致栈溢出
- 客户端在 WebSocket 与 TCP 两条路径上用 catch_unwind 包裹
  decode_message:任何平台解码器 panic 都降级为可跳过的解码错
  误,连接与录制继续存活

新增 5 项回归测试:零长度帧、短长度帧、合法帧后跟畸形头、单层
zlib 嵌套仍可解码、超深嵌套被丢弃。
2026-08-30 15:03:59 +00:00
Noah 01a33213b8 ci: 修复发布与 CI 工作流的权限、死代码及供应链问题 (#1679)
release.yml:
- Windows 归档条件比较 matrix.os 与 windows-2019 永远为假,实际
  依赖 MSYS 的 .exe 兜底才能工作,改为按 runner.os 判断
- publish 归档循环中 exe 变量不重置,目录顺序变化时会污染后续平台
- 替换已归档的 actions-rs/toolchain、actions-rs/cargo,改用
  dtolnay/rust-toolchain 与 cross;checkout@v2(node12 已下线)升 v4
- 弃用的 ::set-output 改为 GITHUB_OUTPUT
- 顶层收紧为 contents: read,仅 publish 任务授予写权限

ci.yml:
- tag 触发从 '*' 收紧为 v[0-9]+.[0-9]+.[0-9]+,任意 tag 不再能
  触发 PyPI 正式发布
- setup-node@v3 升 v4

docker-publish.yml:
- 顶层 packages: write 会授予 pull_request 触发的 test 任务,收紧
  为顶层只读、build/merge 任务单独授予
- 移除 buildx 已弃用的 install: true;docker-compose v1 命令改为
  compose v2 子命令

docs-publish.yml:补 paths 过滤与显式权限。
Termux-publish.yml:setup-node@v3 升 v4。

新增 test.yml:PR 与 master push 运行
cargo test -p biliup -p biliup-cli -p danmaku,合并前有真实测试门禁。

Dockerfile:FFmpeg 从滚动的 latest tag 改为固定 autobuild 版本并
强制 SHA-256 校验,下载被篡改或损坏时构建失败。

docker-compose.yml:占位符 <HOST_PORT> 等使解析必然失败,改为
带默认值的环境变量;移除已废弃的 version 字段。

新增 .dockerignore:排除 .git、构建产物及本地数据库/凭据/日志,
避免进入构建上下文。
2026-08-30 15:00:16 +00:00
Noah 8a19c05c06 fix(webui): 注册提交前执行表单校验,确认密码规则不再被绕过 (#1677)
登录页的提交按钮直接调用 handleSubmit 发起 fetch,从不触发
Semi Form 的规则校验:确认密码字段的一致性 validator 从未执行,
注册时确认密码留空或与密码不一致都能成功提交。

现在提交前先 formApi.validate():校验失败时终止提交,错误信息
展示在对应字段下方;同时为确认密码补充 required 规则,留空时
给出明确提示而不是笼统的“两次密码输入不一致”。
2026-08-30 14:56:56 +00:00
Noah 631e1160b1 fix(flv): 使用 write_all 保证 FLV 标签与 JSON 输出完整写入 (#1676)
flv_writer 中三处使用 io::Write::write 写入数据:write 允许部分
写入且把已写字节数作为 Ok 返回,短写时 previous_tag_size 等
字段会静默缺字节,产出的 FLV 标签结构损坏,播放器与解析器
在该点之后无法继续解析。

将 write_tag 的 previous_tag_size 写入、write_previous_tag_size
与 to_json 的换行写入全部改为 write_all,公开签名保持不变。
新增短写 Writer 回归测试,验证 4 字节 previous_tag_size 在部分
写入场景下仍被完整写出。
2026-08-30 14:54:09 +00:00
Noah 2c3592f117 fix(bilibili): 稿件分页查询复用登录客户端,不再每页新建 Client (#1675)
archives() 之前在分页循环中每请求一页都重新 build 一个
reqwest::Client 和 cookie jar:连接无法复用,代理配置丢失导致
查询绕过 --proxy 直连;且 Jar::add_cookie_str 会把 "; " 拼接的
整串 Cookie 当作单条 Set-Cookie 解析,实际只有第一个 Cookie
生效,其余键值被当作 Cookie 属性丢弃。

改为复用登录时构建的 self.client:cookie store 已含全部登录
Cookie,UA 与原实现一致,并保留代理配置与连接池,同时按原
行为为该请求保留 60 秒总超时。
2026-08-30 14:52:47 +00:00
Noah 5c6c720fdc fix(upload): 测速时单条线路失败不再中断整体测速 (#1674)
Probe::probe 对每条线路的 ping 使用 ? 传播错误,任意一条线路
DNS/连接/读取失败都会让整个测速立即返回错误,即使后续线路
完全可用,调用方也会因 unwrap_or_default 直接退回默认线路。

现在单条线路网络错误或异常状态码只记录警告并跳过该线路,
继续测速其余候选;全部线路失败时返回统一错误。
2026-08-30 14:51:09 +00:00
NoahandNoah 616432d190 feat(sync): 恢复边录边传下载器的流式上传管线 (#1673)
Rust 重写后 sync-downloader 被静默映射为 stream-gears 落盘下载,
边录边传完全失效(#1612)。本次恢复 ffmpeg stdout 到 UPOS 流式
分片上传、逐段投稿/追加分 P 的完整链路,并修复可靠性问题:

- UPOS 拆分为上传 parts 与显式 complete 两阶段,parts 按
  partNumber 排序并校验连续,complete 带状态检查与重试
- 录制同时写临时文件(有界通道背压);短流/预传失败时按实际
  长度回退为文件重传,投稿成功后清理临时文件
- SyncSession 跨拉流重连保留 aid,分 P 严格按录制顺序提交,
  首投结果不确定时停止自动重试避免重复建稿
- 上传/投稿错误向上传播;外层重试只在分 P 推进后重置退避
- 修复 www.douyin.com/live/{id} 被解析成 web_rid=live 的问题

Co-authored-by: Noah <17682131+ransxd@users.noreply.github.com>
2026-08-30 14:50:37 +00:00
yjywybb 051f5c7eb0 fix(upload): preserve filename-invalid characters in submission titles (#1671) 2026-08-25 07:50:23 +00:00
ForgQi 03b7a84f55 Allow remote Web UI bootstrap with auth
This change updates the server default bind to 127.0.0.1 for safer local-only access while documenting how to expose the Web UI via --bind 0.0.0.0 --auth for remote or Docker use. It also removes the previous restriction that blocked first-time admin bootstrap on non-loopback addresses, allowing authenticated remote initialization and bumping the workspace version to 1.2.4.
2026-08-24 01:13:29 +08:00
NoahandNoah c351c39771 fix(server): 会话 Cookie 不再强制 Secure,修复 HTTP 远程访问登录后无法保持登录态 (#1669)
Co-authored-by: Noah <17682131+ransxd@users.noreply.github.com>
2026-08-23 15:20:11 +00:00
NoahandNoah 6cef8afe22 fix(stream-gears): 登录函数释放 GIL 后再执行阻塞网络请求 (#1667)
Co-authored-by: Noah <17682131+ransxd@users.noreply.github.com>
2026-08-22 19:25:48 +00:00
ForgQi eb8699b203 Add recording policy and time-range checks
This change adds a recording policy layer that enforces time-window eligibility and excluded title keywords before probing and before recording. It updates the streamer API to surface reasons like OutOfSchedule and TitleExcluded, clamps ffmpeg/segment durations to the configured window, and exposes the last rejection reason from the worker state for the frontend.
2026-08-22 19:15:48 +08:00
NoahandNoah 1affe97d87 fix(webup): Data 声明 desc_v2 字段,避免提交时被 asdict 静默丢弃 (#1666)
Co-authored-by: Noah <17682131+ransxd@users.noreply.github.com>
2026-08-22 08:52:05 +00:00
cls3389 8fe338b5f5 fix(douyin): 下播后回放房间仍被误判为直播中 (#1663)
抖音主播下播后,直播间跳转 reflow 回放页,接口仍返回 status=2
和可拉取的回放流,导致 monitor 循环误判开播并反复录制垫片流。

两处独立防线:
- get_room_info: finish_time > 0(抖音已写入直播结束时间)判定未开播
- select_stream_url: 流 URL 含 rtm_expr_tag=reflow_room_info
  (回放流标记)时拒绝作为直播流返回

Fixes #1532
2026-08-22 08:39:50 +00:00
NoahandNoah 3750ced583 fix(hls): respect playlist refresh lifecycle (#1665)
Co-authored-by: Noah <17682131+ransxd@users.noreply.github.com>
2026-08-22 08:37:05 +00:00
Noahandransxd e1c4b6887e fix(downloader): 纯视频流分段时避免序列头 expect 导致的 panic (#1664)
分段重建逻辑此前对 on_meta_data / aac_sequence_header / h264_sequence_header 使用 expect。纯视频流(无音频,aac_sequence_header 恒为 None)在首次分段时必然 panic,导致 7x24 录制任务中断。改为缺失时跳过并告警,不再中断录制。

新增回归测试 pure_video_stream_segments_without_panic:构造仅含 onMetaData 与 H264 序列头、无音频标签的 FLV 流并触发分段;修复前该测试会 panic(aac_sequence_header does not exist),修复后通过。

Co-authored-by: ransxd <ransxd@users.noreply.github.com>
2026-08-22 08:35:19 +00:00
Mostly_Harmless b438d72415 feat(web): 完善账号管理、稿件查询与安全边界 (#1660)
启动 Web 服务时自动登记 CLI 指定的 B 站凭据,并为账号登记、刷新和删除统一补上并发去重、原子写入及引用检查。

同时收紧默认监听、管理员初始化、认证路由、静态文件和日志 WebSocket,恢复 server 的 ds_update.log,并避免凭据及上游响应进入日志。

新增按账号查询远程稿件的 API 与 Web 页面,补齐严格分页和响应校验。

关联 biliup/biliup#1624

关联 biliup/biliup#1658

关联 biliup/biliup#724
2026-08-16 14:05:20 +00:00
vector0902androotuc ccb016be4f fix(upload): include lock file path in rate-limit lock error message (#1661)
Co-authored-by: rootuc <rootuc@notunixchan>
2026-08-15 06:40:18 +00:00
PingouinFerreux 0c3bbe2ec1 fix: switch star history chart to a working provider (#1659)
The star history chart in the README is no longer loading because the upstream API it depended on stopped working. Point the chart and its link to the star-history service hosted at star-history.dera.page, which uses a different data source and requires no API key, so the chart renders again for everyone.

This keeps the same query and hash parameters as before, so only the domain in the two URLs changes.
2026-08-14 07:59:58 +00:00
dreammu 3e8b55c2c4 fix(upload): update upload lines (#1656) 2026-08-07 09:28:12 +00:00
Micuks adf6a1c03b fix(hls): 选取最高画质的流,而非固定取 variants[0] (#1649)
HLS 下载器一直取 master playlist 里的第一个变体。Twitch 会把各档
转码清晰度排在源流前面,所以经常录到较低画质的转码流,而不是源流。

改为选取带 RESOLUTION、且码率最高的变体,并跳过 I-frame(trick-play)
流。若没有变体带分辨率,则回退到码率最高的非 I-frame 变体,最后回退
到第一个变体,保证在异常 playlist 下行为不退化。
2026-07-14 14:35:35 +00:00
ForgQi 03c36e1da7 Add batch live-check support + fixes
Introduce batch live-check infrastructure and implementations. Add BatchCheckRequest and LivePlugin::supports_batch_check/batch_check API; Monitor uses BatchLiveCache and batch_verdict to prefilter rooms (platform-level checks + caching and fast rotation). RoomsActor gains PlatformUrls. Implement batch_check for Picarto (explore pagination) and Twitch (GQL batching with auth retry). Other fixes: Douyu auth-failure detection and key invalidation + retry, Inke optional response handling, streamlink runtime tweaks for yt-dlp/Twitch ffmpeg, small test additions and import refactors.
2026-07-14 21:33:23 +08:00
ForgQi 083313257f Improve downloader reliability and add caching
- Add process-level caching for expensive API calls: login tokens (Afreecatv, Douyin), encryption keys (Douyu), room IDs (Douyu, Huya)
- Implement Huya mini-app API option (huya_mobile_api) with WUP protocol support for getCdnTokenInfoEx
- Add cdn_fallback for Huya with health checks and protocol negotiation
- Improve Bilibili stream selection with protocol fallback (hls_fmp4→stream) and qn-based grouping
- Fix Twitch auth token invalidation handling and improve cookie file management
- Refactor YouTube entry selection with proper playlist recursion and date filtering
- Add retry logic for network failures (Douyu betard) and improve error recovery
- Share WBI signer state across Bilibili check_stream calls to respect 2-hour key cache interval
- Always pass --download-archive for YouTube (including live) to prevent post-stream duplicates
- Add field aliases (inke: live_name, kilakila: backPic) and fix Niconico suffix (flv→ts)
2026-07-14 00:32:29 +08:00
ForgQi 18c5bf086e Define AT_HWCAP2 for ppc64le; enable tokio
Set CFLAGS_powerpc64le_unknown_linux_gnu=-DAT_HWCAP2=26 in CI to work around aws-lc-sys (via rustls) calling getauxval(AT_HWCAP2) on manylinux2014 where glibc 2.17 headers lack AT_HWCAP2; glibc still provides getauxval so the value is safe and only used for ppc64le cc invocations. Also enable the tokio "io-std" feature for the biliup-cli crate to opt into the io-std I/O implementation.
2026-07-11 03:53:51 +08:00
ForgQi ddc29f63b4 Update deps & fix server upload/download
Bump dependencies (Cargo.toml + Cargo.lock) to newer crate versions and adapt code to API changes. Refactor server-side upload/download logic, the upload_streamer model, and related API endpoints to improve streaming handling, buffer/error propagation and compatibility with updated crates (e.g. tokio-tungstenite/futures/etc.). These changes align the codebase with the upgraded dependency set and fix streaming/endpoint issues introduced by those upgrades.
2026-07-11 01:57:48 +08:00
ForgQi 58dc75b6ca Build free-threaded wheels for Python 3.14t (#1648)
maturin 1.14 dropped support for free-threaded Python 3.13
("Free-threaded Python interpreter is only supported on 3.14 and
later"), which broke the linux/musllinux/macos free-threaded wheel
steps since maturin-action always uses the latest maturin. Switch
the free-threaded builds from python3.13t to python3.14t, matching
what maturin generate-ci now emits.
2026-07-10 17:08:08 +00:00
Micuks 33649fb568 fix server submit_api web routing (#1645) 2026-07-10 15:11:39 +00:00
Aluneu 165d4eac68 Update CHANGELOG.md (#1646)
更新发行版信息
2026-07-05 11:08:39 +00:00
XuanRan_Nyan 08b96b73f8 feat(douyu): build huos url for forced hs (#1644) 2026-06-26 11:59:42 +00:00
dplei 0fe20687a3 fix(upload): fall back to live URL when copyright_source is blank (#1640)
When the upload template's reprint source (copyright_source) is left
empty, the web form submits an empty string rather than null, so it is
stored as Some(""). build_studio used `unwrap_or_else`, which only fires
on None, so the submitted source became an empty string instead of the
live room URL — contradicting the UI hint
"如不填写转载来源默认为直播间地址".

Treat None and whitespace-only values as "unset" and fall back to the
live room URL. Extract the resolution into a pure `resolve_source`
helper with unit tests covering none / empty / whitespace / real value.
2026-06-11 02:54:59 +00:00
ForgQi b2ab974431 Update README with new features and cleanup
Reorganize features and remove command line tool section.
2026-06-08 01:32:00 +08:00
ForgQi b5ddff0564 Refactor Python package and simplify entry
Remove legacy engine and many plugin modules, refactor upload plugins to standalone classes, and simplify package entry points. __init__.py now provides a safe __version__ fallback; __main__.py directly invokes stream_gears.main_loop. BiliWeb/BiliWebAsync plugins were adapted to use a FileInfo NamedTuple and no longer depend on engine.UploadBase/Plugin decorator. Documentation updated (README expanded with new commands and CLI/SKILL integration) and a new SKILL.md added. Workspace version bumped to 1.2.1 in Cargo.toml (lock updated accordingly).
2026-06-06 03:22:54 +08:00
ForgQi 431d1f61ab Support optional segment limits and danmaku files
Make file_size and segment_time optional and normalize/validate their values so segmentation can be disabled. Frontend: send nulls for empty fields and allow empty segment_time via validation; add showClear to inputs. Downloader: extend SegmentInfo with optional danmaku_file_path and propagate it from various downloaders; DanmakuClient::rolling now returns a bool to signal whether a danmaku file was produced. Uploader: add segment_paths helper to include danmaku files in upload/postprocessing, and preserve processed paths; add unit tests. HookStep: stream child process stdout/stderr as raw bytes to terminal and append to a log file (tee), and simplify run/move logic to use explicit path lists. Cargo.toml: bump reqwest to 0.13.4. Tests and minor adjustments added throughout to support these changes.
2026-06-04 01:19:27 +08:00
ForgQi a529bbcbd2 Update lib.rs 2026-06-01 02:20:59 +08:00
ForgQi 3655679a58 Refactor live downloader architecture
Migrate the server-side live downloading to the new biliup::downloader::live API and simplify the actor model. Key changes:

- Introduce a new server/core/live module that builds LiveRequest/LiveOptions/LiveCredentials, maps runtimes, and creates danmaku clients.
- Replace legacy plugin/DownloadBase actor flow with LivePlugin-based workflows and a start_download_workflow entrypoint; monitor now acquires a Semaphore permit before starting a download to enforce concurrency.
- Remove the old plugin module and per-platform plugin plumbing; many platform implementations were moved/renamed under crates/biliup/src/downloader/live.
- Adjust DownloadManager, Monitor, and common download/upload code to use LivePlugin, LiveStream and related types; pass uploader channel and download slot semaphore into Monitor.
- Add download_stream helper to support direct FLV/HLS downloads via StatelessClient, and extend Streamlink downloader to support Niconico credentials/options.
- Update usage of context/streamer info and danmaku initialization to the new LiveStream-based model.
- Bump workspace version to 1.2.0 (Cargo.toml) and related package versions.

This refactor centralizes live logic on the shared biliup downloader live API, simplifies concurrency control, and removes the previous actor-based plugin indirection.
2026-06-01 01:50:00 +08:00
ForgQi 219334f936 fix ci 2026-05-30 02:24:52 +08:00
ForgQi df898c5d35 Update Dockerfile 2026-05-30 01:37:52 +08:00
ForgQi bde581fa07 Update Dockerfile 2026-05-30 01:27:28 +08:00
ForgQi c395b31148 Replace Python Danmaku with Rust implementation
Remove the legacy Python biliup/Danmaku package and related douyin utilities, and adjust PyInstaller spec to stop bundling those resources. Introduce and expand Rust-based danmaku and plugin implementations (crates/danmaku, crates/biliup-cli server/core/plugin/*, and stream-gears changes) and update various server/downloader modules to integrate them. Also add a TOML dependency to Cargo.toml to support the new Rust changes.
2026-05-30 01:17:18 +08:00
XenoAmess 62bd16de8e chore(doc): update CHANGELOG.md and other docs (#1631) 2026-05-28 02:02:30 +00:00
方程 697a525b55 Add comment listing and reply commands (#1628) 2026-05-25 06:38:50 +00:00
Micuks b2b92e6fee feat(hook): 启用 segment_processor 阶段,新增 remux:mp4 内建钩子 (#1626)
* feat(hook): built-in remux:mp4 hook + activate segment_processor stage

B站 transcoder rejects raw .ts uploads with PTS jumps ("您的视频未能成功
转码 ... 该视频内部存在时间戳跳变"), which is common on HLS reconnects
mid-segment. The fix is to feed it an mp4 with monotonic PTS — a plain
ffmpeg copy-remux is enough.

Add a built-in `HookStep::Remux { remux: String }` variant that runs
`ffmpeg -fflags +genpts+igndts -c copy -bsf:a aac_adtstoasc
-movflags +faststart -avoid_negative_ts make_zero` on .ts/.m2ts inputs.
Lossless and idempotent (reuses existing .mp4 sibling). Currently
target="mp4" only.

To make this useful, segment_processor (defined in config but never
previously invoked) is now executed in pipeline_upload_videos BEFORE
each upload_single_file. That's the right stage for ts→mp4 — must
run before pre_upload, not after.

Path-mutating processor pipeline:
- new execute_paths(&mut Vec<PathBuf>) API on HookStep
- new process_video_paths helper
- Remux replaces .ts entries with the new .mp4 entries so downstream
  stages (upload, postprocessor) see the right paths.
The legacy execute(&[&Path]) path stays for postprocessor-style steps
that don't mutate the working set.

Config form:

  segment_processor:
    - {remux: "mp4"}

Tests cover YAML/JSON deserialization of all 4 HookStep variants
and Remux skipping non-.ts inputs.

* fix(upload): 单段 segment_processor / upload 失败不再拖死整批

之前 pipeline_upload_videos 里两个 ? 把单段失败直接抛到
process_with_upload,让后续 submit + execute_postprocessor(含 rm)
全部跳过。在磁盘紧的场景下,ffmpeg remux 写 mp4 header 时遇到
"No space left on device" → segment_processor Err → 整个直播会话已
成功上传的段也不会触发本地 rm → 文件堆积 → 磁盘更紧 → 下一段又失败,
正反馈把磁盘吃完。

把两处 ? 改成 log + continue:
- segment_processor 失败:跳过当前段,下个事件继续
- upload_single_file 失败:跳过,已上传的段照常进入 uploaded.{videos,paths}

直播会话结束(channel close)时,submit_to_bilibili 仍然能用收集到
的 videos 提交,execute_postprocessor 仍能 rm 已成功的本地文件。

修复对应残留 1.6 TB 历史文件的根因(35 次/5 天 "Process segment
event failed: ffmpeg remux failed ... No space left on device")。

* test(hook): remux 在 ffmpeg 失败时返回 Err(供 pipeline 容错前置条件)

复现 pipeline_upload_videos 容错路径的触发条件:segment_processor
(Remux step) 在 ffmpeg 退非 0 时返回 Err。生产事故的具体表现就是磁盘
满让 ffmpeg 写 mp4 header 失败,调用方必须能从该 Err 优雅恢复
(log + continue),否则整批已上传的段都拿不到 postprocessor 的 rm。

完整 pipeline_upload_videos 的端到端测试要 mock UploadContext
(BiliBili client + 登录态),这是后续重构 seam 的方向。
2026-05-21 02:15:23 +00:00
XuanRan_Nyan a24d46aae7 refactor: Implementing the Danmaku client in Rust (#1570)
* refactor: Implementing the Danmaku client in Rust

* test
2026-05-18 04:16:23 +00:00