feat: 发布 0.6.21，社区共享库改为项目自带社区主站，修复 0.6.20 共享不可用；自检补 IPv6 / FastTrack
- 社区主站（SHARE_HUB=true），默认地址 https://rosdpi.suyijun.top:8443；实例只需出站访问，不需要公网 IP - 令牌注册、按来源网络（IPv4 /24、IPv6 /56 加盐哈希）计票，≥2 个网络一致且领先才发布 - ETag/304、本地缓存、限频、压缩炸弹防护；主站投票进程内计入，不再经 NAT 回环 - 修复 0.6.20：部署量接口 500、共享任务从未运行、开关保存不上；删除 WebDAV / 旧中转代码 - 接入自检：缺 IPv6 嗅探规则提示并给命令；FastTrack 已限定 connection-bytes 时不再误报 - 重算历史分类请求超时放宽到 10 分钟；新增 GET 接口烟雾测试与主站端到端测试 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Su YijunandClaude Opus 5.5 committed
1 parent
f16448d4cb
commit
b0fcb7d20e
22 files changed
+1054
-403
No files matched your search
+6
-5
@@ -75,8 +75,9 @@ LAN_CIDRS=192.168.88.0/24
|
||||
# RETENTION_EVENT_DAYS=180 # 审计事件、操作日志
|
||||
# 以上留存天数也可以在网页 系统设置 → 日志与留存 里改，网页优先
|
||||
|
||||
# ---------- 社区共享库（debug16） ----------
|
||||
# 把本网络实测的「远端 IP → 应用」匿名投票上报给社区中转，并拉取社区库；只含远端 IP 和应用 ID，
|
||||
# 不含本机终端、域名记录、自定义应用，私网地址剔除。总开关在网页 系统设置 → 外部数据源（默认打开）。
|
||||
# SHARE_URL=https://<社区中转地址> # 留空不参与；中转可以是任何一个开启了「共享中转」的实例
|
||||
# SHARE_TOKEN= # 中转设置了 SHARE_TOKEN 时才需要
|
||||
# ---------- 社区共享库 ----------
|
||||
# 把本网络实测的「远端 IP / 域名 → 应用」匿名投票汇总到项目自带的社区主站，并拉取社区库；不含本机终端、
|
||||
# 访问记录、自定义应用，私网地址剔除。开关在网页 系统设置 → 外部数据源（默认打开）。
|
||||
# SHARE_HUB_URL=https://rosdpi.suyijun.top:8443 # 社区主站（内置，一般不用改）；自建私有社区时改成自己的主站
|
||||
# SHARE_HUB=false # 只有主站自己设为 true
|
||||
# SHARE_VERIFY_TLS=true # 私有主站用自签证书时才设 false
|
||||
@@ -1,5 +1,30 @@
|
||||
# Changelog
|
||||
|
||||
## [0.6.21] - 2026-10-25
|
||||
|
||||
社区共享库重做：项目自带社区主站，不再依赖 WebDAV / 第三方存储（debug13~22 复查）
|
||||
|
||||
- **修复 0.6.20 的共享整体不可用**：
|
||||
- 设置页打开就报 500：部署量接口调用了已删除的函数；
|
||||
- 共享任务一次都没跑过：调用了已删除的 `sharing_on`，启动时报错被吞掉；
|
||||
- 共享开关保存不上：前端没把开关发给后端。
|
||||
- **社区主站**（debug22 改为项目内实现）：主站和普通实例是同一套程序，主站部署设 `SHARE_HUB=true`，地址内置为 `https://rosdpi.suyijun.top:8443`（`SHARE_HUB_URL` 可改成自建的私有社区）。
|
||||
- 普通实例每 30 分钟心跳、每 6 小时附带一次匿名投票，拉社区库时没变化主站回 304；社区库缓存在本地，主站暂时不可达时继续可用。
|
||||
- 主站自己的投票在进程内计入，不再通过公网地址访问自己（旧中转在 NAT 回环不通时永远同步不上）。
|
||||
- 设置页只保留“参与社区共享”开关，显示主站地址、连接状态和失败原因、社区库规模、全球已部署 / 在线 / 离线；主站还显示来源网络数，反向代理没传客户端 IP 时提示。
|
||||
- **防冒充、防刷票**：
|
||||
- 每个实例注册时领取令牌，别人无法冒充它或覆盖它的投票；
|
||||
- 按**来源网络**计票（IPv4 /24、IPv6 /56，只存加盐哈希、不存来源 IP），至少 2 个不同网络的实例投给同一应用、并且票数领先才发布——同一网络伪造再多实例也只算一票；
|
||||
- 同一网络每天限注册 5 个实例，投票每小时最多收一次，请求体和解压后大小都有上限（防压缩炸弹）；
|
||||
- 应用白名单、域名格式、私网地址剔除照旧；主站证书恢复校验（`SHARE_VERIFY_TLS`，私有社区用自签证书时才关）。
|
||||
- **不需要公网 IP**：心跳、上报、拉库都是实例主动连主站。主站目前只有 IPv6 地址：Linux 一键安装直接可用；Docker 默认网桥没有 IPv6 出口，`docker-compose.yml` 末尾给了开启方法（Docker 27+），连不上时设置页直接提示原因。
|
||||
- 删除 WebDAV 共享和 0.6.15~0.6.19 的中转代码。旧中转留下的无令牌心跳不计入部署量，各实例升级后会重新注册。
|
||||
- **接入自检**：
|
||||
- 路由器有 IPv6 公网地址、嗅探规则却只有 IPv4 的一份时提示，并给出 `/ipv6 firewall mangle` 的 4 条命令；
|
||||
- 读取 `/ip/firewall/filter`，FastTrack 规则已经限定 connection-bytes 时不再误报，没限定时直接给出修改命令。主站点实测：FastTrack 开着、SNI 只解析到几十个握手，“国内网络”占 21%，主要原因就在这里。
|
||||
- 重算历史分类的请求超时从 20 秒放宽到 10 分钟（审计报告 P1-3：实测 80 秒，前端会误报失败）。
|
||||
- 新增接口烟雾测试（所有无路径参数的 GET 接口都不能返回 500），以及主站 / 客户端端到端测试。
|
||||
|
||||
## [0.6.20] - 2026-10-25
|
||||
|
||||
社区共享库改用 WebDAV（NAS 自带），取消中转模式（debug22）
|
||||
|
||||
+4
-1
@@ -36,6 +36,7 @@ services:
|
||||
|
||||
## What's new
|
||||
|
||||
- **0.6.21**: 社区共享库重做——项目自带社区主站，不再需要 WebDAV / 第三方存储：令牌注册、按来源网络计票（≥2 个不同网络一致才发布，防伪造实例刷票）、ETag 增量拉取、本地缓存；修复 0.6.20 部署量接口 500、共享任务从未运行、开关保存不上。接入自检新增 IPv6 嗅探规则检查，FastTrack 已限定 connection-bytes 时不再误报，并直接给出修改命令。Community library rebuilt around a built-in hub (no WebDAV/third-party storage): per-instance tokens, votes counted per source network, ETag pulls, local cache; fixes the 0.6.20 deployments 500 and the never-running share job. Self-check now flags missing IPv6 sniff rules.
|
||||
- **0.6.12 / 0.6.11 / 0.6.10 / 0.6.9 / 0.6.8**: 自学习与共享系列。**本地自学习库**：本网络实测的「IP → 应用」（IPv6 按 /64 前缀聚合）与「域名 → 应用」长期记忆，每小时重建，DNS 缓存过期、DoH、裸 IP 流量都能按记忆归类；**社区共享库**：各部署实例把实测结论匿名共享（中转聚合、至少 2 个实例见过才发布），本地实测优先、社区只补空白；**全球部署量**：实例心跳上报，中转公开已部署 / 在线 / 离线数量；共享总开关默认打开；**一键自更新**：Linux 二进制部署在网页上检测并一键升级（自动下载、替换、重启服务）；**Linux 单文件二进制部署**：`install.sh` 一键安装 systemd 服务（SQLite 零配置起步）；修复落库卡死（代理长域名撑爆 access_log.remote 列导致每分钟落库失败、在线终端归零）；组播 / 广播流量不再计入统计；17 个 DPI 大类配色互异。
|
||||
- **0.6.7**: fixes for the 0.6.5 / 0.6.6 history reclassification and rule auto-registration. Reclassification now only touches days before today and only upgrades vague rows (HTTPS / Chinese websites / unknown / CDN) to a specific app — it no longer downgrades apps identified live by IP range, port or ASN. New upstream blackmatrix7 directories are downloaded first and then assigned by a vote of the existing libraries. The manual dark theme now uses the 17 distinct group colors, and the AdGuard Home password is masked in the operation log.
|
||||
- **0.6.6**: AdGuard Home's newer query-log format (`{"data": [...]}`) is now read (0.6.5 always got 0 answers); new upstream rule directories / lists are registered automatically; history reclassification looks up the DNS cache for IP-only rows and runs automatically after library updates.
|
||||
@@ -119,7 +120,9 @@ Open `http://<host>:8080`. The setup wizard has two steps. 打开网页后按两
|
||||
| `ADMIN_PASSWORD` | empty | initial password for `admin` |
|
||||
| `ROS_URL` / `ROS_USER` / `ROS_PASSWORD` | empty | RouterOS REST + read-only user |
|
||||
| `SINGBOX_URL` / `MOSDNS_URL` / `ADGUARD_URL` | empty | external sources (Clash API, MosDNS audit, AdGuard Home query log) |
|
||||
| `SHARE_URL` / `SHARE_TOKEN` | empty | community library relay (anonymous IP→app / domain→app votes) |
|
||||
| `SHARE_HUB_URL` | `https://rosdpi.suyijun.top:8443` | community hub (built in); point it at your own hub for a private community |
|
||||
| `SHARE_HUB` | `false` | run this instance as the community hub (only the hub sets `true`) |
|
||||
| `SHARE_VERIFY_TLS` | `true` | verify the hub certificate; set `false` only for a private hub with a self-signed cert |
|
||||
| `TZ` | `Asia/Shanghai` | statistics time zone (weeks start Monday) |
|
||||
|
||||
Full list: https://github.com/suyijun8182/ros-dpi#环境变量
|
||||
@@ -42,7 +42,7 @@ RouterOS ◀─REST 只读（可选）── 主机名、IP↔MAC、WAN 口速
|
||||
- **域名 → 应用规则库，四层叠加**：
|
||||
1. 公开库：内置手工规则库 + [v2fly](https://github.com/v2fly/domain-list-community)（约 2500 应用）+ [blackmatrix7](https://github.com/blackmatrix7/ios_rule_script)（自动登记上游新目录）+ Hagezi / UT1 分类列表 + 国内域名列表 / 官方 IP 段 / 国内 IP 段兜底；
|
||||
2. **本地自学习库**：本网络实测的「IP → 应用」与「域名 → 应用」长期记忆——DNS 缓存过期、DoH、裸 IP 流量都能按记忆归类，每小时自动重建；
|
||||
3. **社区共享库**：各部署实例把实测结论匿名共享（至少 2 个实例见过才发布），本地没有结论时兜底；
|
||||
3. **社区共享库**：各部署实例把实测结论匿名汇总到项目自带的社区主站（至少 2 个不同网络的实例一致才发布），本地没有结论时兜底；
|
||||
4. 以上都没有时，才归到 HTTPS / 国内网络这类笼统分类。
|
||||
- 终端用 MAC 区分，IP 变化不影响统计。
|
||||
|
||||
@@ -186,7 +186,7 @@ EOF
|
||||
|
||||
1. **公开库**（自动更新）：内置手工库 + v2fly（每天自检更新）+ blackmatrix7（每周，上游新目录自动登记：先下载、再由现有识别库投票定归属）+ Hagezi / UT1 分类列表 + ASN（iptoasn）+ 国内域名列表 / 官方 IP 段 / 国内 IP 段兜底。
|
||||
2. **本地自学习库**：本网络实测的「IP → 应用」（IPv6 按 /64 前缀聚合）与「域名 → 应用」长期记忆，每小时从数据库重建；DNS 缓存过期、DoH、裸 IP 流量都能按记忆归类。每次历史重算自动纠正它。
|
||||
3. **社区共享库**：各部署实例把实测结论匿名共享——只含远端 IP / 域名、应用 ID 和字节票数，不含本机终端、域名记录与自定义应用，私网地址剔除；中转聚合**至少 2 个实例见过**的映射才发布。系统设置 → 外部数据源 填中转地址即参与（总开关默认打开），也可把自己的实例开成中转（公网可达）。中转的 `/api/share/deployments` 公开显示全球已部署 / 在线 / 离线数量。
|
||||
3. **社区共享库**：项目自带社区主站，不依赖任何第三方存储。各实例每 30 分钟向主站心跳、每 6 小时上报一次匿名投票——只含远端 IP / 域名、应用 ID 和字节票数，不含本机终端、访问记录与自定义应用，私网地址剔除；主站按**来源网络**计票（IPv4 /24、IPv6 /56，只存加盐哈希），**至少 2 个不同网络的实例投给同一应用且票数领先**才发布，一个人伪造多个实例也只算一票；每个实例有主站发的令牌，别人无法冒充。社区库缓存在本地，主站暂时不可达时继续可用。各实例**不需要公网 IP**（都是主动连主站），但要能访问主站：主站目前只有 IPv6 地址，Linux 一键安装直接用主机网络即可；Docker 默认网桥没有 IPv6 出口，需按 `docker-compose.yml` 末尾的说明开启（Docker 27+）。系统设置 → 外部数据源 可看连接状态、社区库规模与全球已部署 / 在线 / 离线数量；开关默认打开，关闭后不再上报也不再使用社区库。自建私有社区：一台实例设 `SHARE_HUB=true`，其它实例把 `SHARE_HUB_URL` 指向它（主站要能被各实例访问，建议 HTTPS）。
|
||||
4. 以上都没有时，才归到 HTTPS / 国内网络这类笼统分类。
|
||||
|
||||
识别库有变化或过了零点，自动把今天以前的历史流量按新规则重算（只升级不降级）；识别规则页也可手动执行。世界地图使用 [Natural Earth](https://www.naturalearthdata.com/)（公有领域）简化数据。
|
||||
@@ -204,8 +204,9 @@ EOF
|
||||
| `SINGBOX_URL` / `SINGBOX_SECRET` / `PROXY_IPS` | 空 | 旁路代理 sing-box 的 Clash API（也可在网页填写，网页优先） |
|
||||
| `MOSDNS_URL` | 空 | MosDNS 审计 API 地址 |
|
||||
| `ADGUARD_URL` / `ADGUARD_USERNAME` / `ADGUARD_PASSWORD` | 空 | 自有 DNS 服务器 AdGuard Home 的查询日志（双层 NAT 场景） |
|
||||
| `SHARE_URL` | 空 | 社区共享库中转地址（debug16）：上报本网络实测、拉取社区库；留空不参与 |
|
||||
| `SHARE_TOKEN` | 空 | 中转的上报令牌（中转设置了才需要） |
|
||||
| `SHARE_HUB_URL` | `https://rosdpi.suyijun.top:8443` | 社区主站地址（内置，一般不用改）；自建私有社区时改成自己的主站 |
|
||||
| `SHARE_HUB` | `false` | 本实例作为社区主站（只有主站部署设为 `true`） |
|
||||
| `SHARE_VERIFY_TLS` | `true` | 校验主站证书；私有社区用自签证书时才设 `false` |
|
||||
| `FAKEIP_CIDRS` | `198.18.0.0/15,fc00::/18` | FakeIP 地址段 |
|
||||
| `ASN_DB` / `CHINA_DB` / `IPLIST_DB` / `BM7_DB` / `HAGEZI_DB` / `UT1_DB` / `OUI_DB` | `true` | 各识别库自动下载开关；没有外网时设 `false` |
|
||||
| `WAN_INTERFACE` | 自动 | WAN 接口名，默认按默认路由识别 |
|
||||
@@ -252,7 +253,7 @@ backend/.venv/bin/python scripts/simulate.py --minutes 5
|
||||
- **Flows**: live connections (sing-box connections show their policy group and outbound node), top destinations with a flow map from your location to each country, and proxy routing per sing-box policy group and node.
|
||||
- **Family members / Employees**: pick the Home or Business scenario under **Settings → Scenario & network**; group each person's devices and see their traffic, online time, top apps and behaviour tags for the last 7 days.
|
||||
- **Audit**: access log (CSV export), audit rules (app, domain, time window, traffic, large upload, rogue router, new device) with 14 push-notification channels, daily / weekly reports, operation log, TTL-based shared-access check, and a RouterOS control-command generator.
|
||||
- **Self-learning & community library**: mappings observed on YOUR network (IP → app, domain → app) are kept as long-term memory and classify traffic even after DNS caches expire, with DoH clients, or for bare-IP flows; deployments can share anonymous votes through a relay, and the relay publishes mappings seen by at least two instances plus a global deployment counter.
|
||||
- **Self-learning & community library**: mappings observed on YOUR network (IP → app, domain → app) are kept as long-term memory and classify traffic even after DNS caches expire, with DoH clients, or for bare-IP flows; deployments share anonymous votes through the built-in community hub (no third-party storage); a mapping is published only when instances on at least two different networks agree, and the hub shows global deployed / online / offline counts. Instances need no public IP — they only connect out to the hub.
|
||||
- **Self-update** (Linux binary install): a one-line installer sets up a systemd service with SQLite out of the box; new versions are detected automatically and applied with one click from the UI — no SSH needed.
|
||||
- Chinese and English UI, light and dark themes, works on phones.
|
||||
|
||||
|
||||
@@ -1 +1 @@
|
||||
__version__ = "0.6.20"
|
||||
__version__ = "0.6.21"
|
||||
@@ -42,9 +42,10 @@ class Settings(BaseSettings):
|
||||
adguard_username: str = "" # AdGuard Home 登录用户名；和密码都留空表示免认证（只在内网监听时）
|
||||
adguard_password: str = ""
|
||||
share_enabled: bool = True # 共享识别库总开关（debug16 第 3 条，默认打开）：关掉后不心跳、不上报、不拉取
|
||||
webdav_url: str = "" # 社区共享库的 WebDAV 地址（debug22，NAS 自带）：如 http://192.168.8.10:5005/ros-dpi
|
||||
webdav_user: str = "" # WebDAV 账号；和密码都留空表示免认证
|
||||
webdav_password: str = ""
|
||||
# 社区共享库（debug16→debug22 重做）：项目自带的社区主站，不依赖第三方存储
|
||||
share_hub_url: str = "https://rosdpi.suyijun.top:8443" # 社区主站地址；自建私有社区时改成自己的主站
|
||||
share_hub: bool = False # 本实例就是社区主站（只有主站部署设为 true）：接收上报、发布社区库与全球部署量
|
||||
share_verify_tls: bool = True # 校验主站证书；私有社区用自签证书时才设 false
|
||||
oui_db: bool = True # 自动下载 IEEE 的 MAC 厂商库（OUI，识别终端类型用，每 30 天更新）
|
||||
asn_db: bool = True # 自动下载 iptoasn 的 IP→ASN 库（公有领域，每天更新）
|
||||
china_db: bool = True # 自动下载国内域名列表（dnsmasq-china-list）与国内 IP 段（每天更新）
|
||||
|
||||
@@ -19,7 +19,7 @@ import httpx
|
||||
from app.config import get_settings
|
||||
|
||||
KEYS = ("singbox_url", "singbox_secret", "proxy_ips", "mosdns_url", "adguard_url", "adguard_username",
|
||||
"adguard_password", "webdav_url", "webdav_user", "webdav_password")
|
||||
"adguard_password")
|
||||
|
||||
|
||||
def effective(db_values: dict) -> dict:
|
||||
|
||||
+16
-14
@@ -17,6 +17,7 @@ from app.models import Setting
|
||||
from app.ros_poller import poller
|
||||
from app import audit # noqa: F401 挂上审计（engine.audit）
|
||||
from app.routers import admin, auth, setup, stats
|
||||
from app.routers import share as share_router
|
||||
from app.routers import audit as audit_router
|
||||
from app.routers import compliance as compliance_router
|
||||
from app.routers import people as people_router
|
||||
@@ -48,7 +49,7 @@ def _on_db_ready():
|
||||
（0.6.0 / 0.6.1 的主域名回填出错时，后面的外部数据源配置、自定义规则都没加载，外部数据源显示“未启用”）"""
|
||||
ensure_admin() # 建管理员失败要报出来（安装向导靠这个提示）
|
||||
for name, fn in (("load_state", engine.load_state), ("apply_settings", _apply_saved),
|
||||
("local_db", _refresh_local), ("share_lib", _share_cycle)):
|
||||
("local_db", _refresh_local), ("share_lib", _share_startup)):
|
||||
try:
|
||||
fn()
|
||||
except Exception as e: # noqa: BLE001
|
||||
@@ -76,23 +77,24 @@ def _auto_update_on() -> bool:
|
||||
|
||||
|
||||
def _share_cycle():
|
||||
"""社区共享库（debug22）：心跳（部署量统计）+ 投票共享，走 NAS 的 WebDAV。开关关闭或未配置地址时跳过"""
|
||||
"""社区共享库：普通实例向社区主站心跳、上报、拉库；主站在进程内聚合（见 app/share.py）"""
|
||||
if not database.is_configured():
|
||||
return
|
||||
from app import share
|
||||
with database.session() as db:
|
||||
if not share.sharing_on(db):
|
||||
return
|
||||
saved = {k: (db.get(Setting, k).value if db.get(Setting, k) else None) for k in integrations.KEYS}
|
||||
cfg = integrations.effective(saved)
|
||||
url = cfg.get("webdav_url") or ""
|
||||
if not url:
|
||||
return
|
||||
with database.session() as db:
|
||||
res = share.webdav_cycle(engine, db, url, cfg.get("webdav_user") or "", cfg.get("webdav_password") or "")
|
||||
res = share.cycle(engine, db)
|
||||
db.commit()
|
||||
print(f"[share] 共享库已同步：已部署 {res['deployed']} 台（在线 {res['online']}），"
|
||||
f"社区库 {res['community_ips'] + res['community_domains']} 条", flush=True)
|
||||
if res:
|
||||
print(f"[share] 社区共享已同步{'（本实例是社区主站）' if share.is_hub() else ''}：已部署 {res['total']} 台"
|
||||
f"（在线 {res['online']}），社区库 {res['community_ips'] + res['community_domains']} 条", flush=True)
|
||||
|
||||
|
||||
def _share_startup():
|
||||
"""启动时先用上次缓存的社区库，再在后台跑第一轮同步（不挡住启动）"""
|
||||
from app import share
|
||||
if not share.is_hub():
|
||||
share.load_cached_library()
|
||||
share.kick(_safe(_share_cycle, "share"))
|
||||
|
||||
|
||||
def _auto_update_rules():
|
||||
@@ -210,7 +212,7 @@ async def _unhandled(request: Request, exc: Exception):
|
||||
|
||||
|
||||
for r in (setup.router, auth.router, stats.router, admin.router, audit_router.router, people_router.router,
|
||||
compliance_router.router):
|
||||
compliance_router.router, share_router.router):
|
||||
app.include_router(r)
|
||||
|
||||
|
||||
|
||||
@@ -227,13 +227,17 @@ class ShareEntry(Base):
|
||||
|
||||
|
||||
class ShareDeployment(Base):
|
||||
"""社区部署心跳（debug16）：实例每小时上报一次存在，中转聚合出 全球已部署 / 在线 / 离线。只含版本号，无任何网络数据"""
|
||||
"""社区主站上的实例登记（debug16 / 0.6.21 重做）：注册时发令牌，每次上报更新心跳。
|
||||
全球 已部署 / 在线 / 离线 从这里统计；只含版本号和来源网络的哈希，不存来源 IP"""
|
||||
__tablename__ = "share_deployments"
|
||||
instance = Column(String(36), primary_key=True)
|
||||
version = Column(String(16))
|
||||
first_seen = Column(BigInteger, nullable=False)
|
||||
last_seen = Column(BigInteger, nullable=False)
|
||||
public_url = Column(String(255)) # 中转实例的公开地址（部署量里的可用中转列表用）
|
||||
public_url = Column(String(255)) # 0.6.15~0.6.19 的中转地址，已停用
|
||||
token_hash = Column(String(64)) # 实例令牌的 SHA-256（令牌本身只在实例那边）
|
||||
net = Column(String(16)) # 来源网络（IPv4 /24、IPv6 /56）的加盐哈希：防一人伪造多个实例刷票
|
||||
last_votes = Column(BigInteger) # 最近一次带投票的上报时间
|
||||
|
||||
|
||||
class ShareDomain(Base):
|
||||
|
||||
@@ -9,6 +9,16 @@ import httpx
|
||||
from app.config import get_settings
|
||||
|
||||
|
||||
|
||||
def _global_v6(addr: str | None) -> bool:
|
||||
"""RouterOS /ipv6/address 的 address（带前缀长度）是不是公网地址"""
|
||||
import ipaddress
|
||||
try:
|
||||
a = ipaddress.ip_interface(addr or "").ip
|
||||
except ValueError:
|
||||
return False
|
||||
return a.version == 6 and a.is_global
|
||||
|
||||
class RosPoller:
|
||||
def __init__(self, engine):
|
||||
self.engine = engine
|
||||
@@ -32,6 +42,8 @@ class RosPoller:
|
||||
self.samples: deque = deque(maxlen=1440)
|
||||
self.mangle: list[dict] = [] # 接入自检用：RouterOS 的 mangle 规则（只读）
|
||||
self.ip_settings: dict = {}
|
||||
self.filter_rules: list[dict] | None = None # /ip/firewall/filter（看 FastTrack 规则有没有限定 connection-bytes）
|
||||
self.has_ipv6 = False # 路由器上有没有 IPv6 公网地址
|
||||
self._sample_prev: dict[str, tuple[float, int, int]] = {}
|
||||
|
||||
@property
|
||||
@@ -85,12 +97,16 @@ class RosPoller:
|
||||
extra = {k: self._opt(c, path) for k, path in (
|
||||
("pools", "/ip/pool"), ("servers", "/ip/dhcp-server"), ("dhcpc", "/ip/dhcp-client"),
|
||||
("pppoe", "/interface/pppoe-client"), ("mangle", "/ip/firewall/mangle"),
|
||||
("mangle6", "/ipv6/firewall/mangle"), ("ipset", "/ip/settings"))}
|
||||
("mangle6", "/ipv6/firewall/mangle"), ("ipset", "/ip/settings"),
|
||||
("filter", "/ip/firewall/filter"))}
|
||||
self._apply(ifaces, arp, leases, addrs, routes, nb6, addrs6)
|
||||
self._apply_status(ifaces, addrs, leases, routes, extra)
|
||||
self.mangle = [{**r, "_family": "ipv4"} for r in extra["mangle"] or []] + \
|
||||
[{**r, "_family": "ipv6"} for r in extra["mangle6"] or []]
|
||||
self.ip_settings = extra["ipset"] if isinstance(extra["ipset"], dict) else {}
|
||||
self.filter_rules = extra["filter"] if isinstance(extra["filter"], list) else None
|
||||
self.has_ipv6 = any(_global_v6(a.get("address")) and a.get("disabled") != "true"
|
||||
for a in addrs6 or [])
|
||||
# 全部成功后再更新展示用的状态，失败时保留上一次的结果
|
||||
self.identity = ident.get("name", "") if isinstance(ident, dict) else ""
|
||||
self.resource = resource if isinstance(resource, dict) else {}
|
||||
@@ -280,10 +296,19 @@ class RosPoller:
|
||||
issues.append("no_sni")
|
||||
elif not any(x["kind"] == "sni" and x["protocol"] == "udp" for x in rules):
|
||||
issues.append("no_quic")
|
||||
if self.ip_settings.get("ipv4-fasttrack-active") == "true":
|
||||
# 有 IPv6 公网地址却只有 IPv4 的嗅探规则：走 IPv6 的 DNS 应答和握手全都看不到（国内运营商默认下发 IPv6）
|
||||
if self.has_ipv6 and any(x["family"] == "ipv4" for x in rules) and not any(x["family"] == "ipv6" and not x["disabled"] for x in rules):
|
||||
issues.append("no_ipv6")
|
||||
fasttrack = self.ip_settings.get("ipv4-fasttrack-active") == "true"
|
||||
# FastTrack 规则已经加了 connection-bytes（连接开头几 KB 不走 FastTrack）时握手照样镜像得到，不算问题；
|
||||
# 读不到 filter 表（权限不够时是空列表）时按旧办法提示
|
||||
ft_rules = [r for r in (self.filter_rules or []) if r.get("action") == "fasttrack-connection"
|
||||
and r.get("disabled") != "true"]
|
||||
ft_limited = bool(ft_rules) and all(r.get("connection-bytes") for r in ft_rules)
|
||||
if fasttrack and not ft_limited:
|
||||
issues.append("fasttrack")
|
||||
return {"checked": True, "port": port, "target": ref[0] if ref else None, "rules": rules, "issues": issues,
|
||||
"fasttrack": self.ip_settings.get("ipv4-fasttrack-active") == "true"}
|
||||
"fasttrack": fasttrack, "fasttrack_limited": fasttrack and ft_limited}
|
||||
|
||||
def status(self) -> dict:
|
||||
r = self.resource if isinstance(self.resource, dict) else {}
|
||||
|
||||
@@ -132,9 +132,6 @@ class SettingsIn(BaseModel):
|
||||
adguard_url: str | None = Field(None, max_length=255)
|
||||
adguard_username: str | None = Field(None, max_length=255)
|
||||
adguard_password: str | None = Field(None, max_length=255)
|
||||
webdav_url: str | None = Field(None, max_length=255)
|
||||
webdav_user: str | None = Field(None, max_length=255)
|
||||
webdav_password: str | None = Field(None, max_length=255)
|
||||
rules_auto_update: bool | None = None
|
||||
share_enabled: bool | None = None
|
||||
|
||||
@@ -179,8 +176,6 @@ def _integrations(db):
|
||||
"mosdns_url": cfg["mosdns_url"],
|
||||
"adguard_url": cfg["adguard_url"], "adguard_username": cfg["adguard_username"],
|
||||
"has_adguard_password": bool(cfg["adguard_password"]),
|
||||
"webdav_url": cfg["webdav_url"], "webdav_user": cfg["webdav_user"],
|
||||
"has_webdav_password": bool(cfg["webdav_password"]),
|
||||
"proxy_auto": integrations.singbox.host_ip()}
|
||||
|
||||
|
||||
@@ -229,6 +224,9 @@ def put_settings(req: SettingsIn, request: Request, user: str = Depends(current_
|
||||
_put(db, "rules_auto_update", "1" if req.rules_auto_update else "0")
|
||||
if req.share_enabled is not None:
|
||||
_put(db, "share_enabled", "1" if req.share_enabled else "0")
|
||||
db.commit()
|
||||
from app import share
|
||||
share.kick(share.run_once)
|
||||
if req.wan_interface is not None:
|
||||
_put(db, "wan_interface", req.wan_interface.strip())
|
||||
poller.wan_override = req.wan_interface.strip()
|
||||
@@ -311,8 +309,9 @@ def library(db: Session = Depends(get_db)):
|
||||
|
||||
@router.get("/system/deployments")
|
||||
def system_deployments(db: Session = Depends(get_db)):
|
||||
"""全球部署量 + 可用中转列表（debug19）"""
|
||||
return deployments_with_relays(db)
|
||||
"""社区共享状态 + 全球部署量（设置页 → 外部数据源 → 社区共享库）"""
|
||||
from app import share
|
||||
return share.status(db)
|
||||
|
||||
|
||||
@router.get("/system/update")
|
||||
|
||||
@@ -0,0 +1,89 @@
|
||||
"""社区主站的公开端点（0.6.21 重做，见 app/share.py）。只有 SHARE_HUB=true 的实例开放，其它实例一律 404。
|
||||
|
||||
- POST /api/share/v1/register 首次注册，领取实例 ID 和令牌
|
||||
- POST /api/share/v1/report 心跳 + 投票（Authorization: Bearer <实例ID>:<令牌>，可 gzip）
|
||||
- GET /api/share/v1/library 社区库（ETag / 304，支持 gzip）
|
||||
- GET /api/share/v1/stats 全球部署量：已部署 / 在线 / 离线（所有人可见）
|
||||
"""
|
||||
from fastapi import APIRouter, HTTPException, Request, Response
|
||||
from fastapi.responses import JSONResponse
|
||||
from starlette.concurrency import run_in_threadpool
|
||||
|
||||
from app import database, share
|
||||
|
||||
router = APIRouter(prefix=share.API, tags=["share"])
|
||||
|
||||
|
||||
def _need_hub():
|
||||
if not share.is_hub():
|
||||
raise HTTPException(404, "本实例不是社区主站")
|
||||
if not database.is_configured():
|
||||
raise HTTPException(503, "社区主站的数据库还没就绪")
|
||||
|
||||
|
||||
def _source(request: Request) -> str:
|
||||
return share.client_ip(request.client.host if request.client else "", request.headers.get("x-forwarded-for"),
|
||||
request.headers.get("x-real-ip"))
|
||||
|
||||
|
||||
def _run(fn):
|
||||
try:
|
||||
return fn()
|
||||
except share.ShareError as e:
|
||||
raise HTTPException(e.status, e.detail) from None
|
||||
|
||||
|
||||
@router.post("/register")
|
||||
async def register(request: Request):
|
||||
_need_hub()
|
||||
if int(request.headers.get("content-length") or 0) > share.MAX_BODY:
|
||||
raise HTTPException(413, "请求内容太大")
|
||||
raw = await request.body()
|
||||
ip = _source(request)
|
||||
|
||||
def work():
|
||||
body = share.inflate(raw, request.headers.get("content-encoding"))
|
||||
with database.session() as db:
|
||||
iid, token = share.register(db, share.net_of(db, ip), str(body.get("version") or ""))
|
||||
db.commit()
|
||||
return {"instance": iid, "token": token}
|
||||
return await run_in_threadpool(_run, work)
|
||||
|
||||
|
||||
@router.post("/report")
|
||||
async def report(request: Request):
|
||||
_need_hub()
|
||||
if int(request.headers.get("content-length") or 0) > share.MAX_BODY:
|
||||
raise HTTPException(413, "上报内容太大")
|
||||
raw = await request.body()
|
||||
ip = _source(request)
|
||||
|
||||
def work():
|
||||
from app.engine import engine
|
||||
body = share.inflate(raw, request.headers.get("content-encoding"))
|
||||
with database.session() as db:
|
||||
row = share.authenticate(db, request.headers.get("authorization"))
|
||||
res = share.report(db, row, share.net_of(db, ip), body, engine.classifier)
|
||||
db.commit()
|
||||
return res
|
||||
return await run_in_threadpool(_run, work)
|
||||
|
||||
|
||||
@router.get("/library")
|
||||
def library(request: Request):
|
||||
_need_hub()
|
||||
with database.session() as db:
|
||||
lib = share.library(db)
|
||||
headers = {"ETag": lib["etag"], "Cache-Control": "no-cache"}
|
||||
if request.headers.get("if-none-match") == lib["etag"]:
|
||||
return Response(status_code=304, headers=headers)
|
||||
if "gzip" in (request.headers.get("accept-encoding") or "").lower():
|
||||
return Response(lib["gzip"], media_type="application/json", headers={**headers, "Content-Encoding": "gzip"})
|
||||
return Response(lib["json"], media_type="application/json", headers=headers)
|
||||
|
||||
|
||||
@router.get("/stats")
|
||||
def stats():
|
||||
_need_hub()
|
||||
with database.session() as db:
|
||||
return JSONResponse(share.stats(db), headers={"Access-Control-Allow-Origin": "*"})
|
||||
+518
-191
@@ -1,48 +1,76 @@
|
||||
"""社区共享库（debug16→debug22）：通过 **WebDAV 文件夹**（飞牛 / 群晖等 NAS 自带）共享本地自学习库。
|
||||
"""社区共享库（debug16 → 0.6.21 重做）：各部署实例互相共享本地自学习库的实测结论，项目自带社区主站，不依赖第三方存储。
|
||||
|
||||
架构（无中心服务、无中转）：
|
||||
- 每个实例把各自的匿名投票写到 WebDAV 上的两个文件（文件名是随机 UUID，不含任何可识别信息）：
|
||||
deployments/<uuid>.json {"version", "ts"} —— 部署心跳（全球部署量统计）
|
||||
votes/<uuid>.json {"entries": [IP→应用], "domains": [域名→应用]}
|
||||
- 每 30 分钟：PUT 自己的两个文件（WebDAV 天然按文件去重，不会互相覆盖），然后列出文件夹里
|
||||
所有其它实例的文件、全部拉下来在本地聚合——**至少 2 个不同实例见过的映射**才进入社区层。
|
||||
- 本地实测优先：社区条目只补本地没有结论的 IP / 域名。
|
||||
架构：
|
||||
- 社区主站：和普通实例是同一套程序，部署时设 SHARE_HUB=true（只有主站这样设）。它接收各实例的上报，
|
||||
聚合后发布社区库与全球部署量（/api/share/v1/*，见 routers/share.py）。主站自己的投票在进程内直接计入，
|
||||
不经过网络（之前的中转要通过公网地址访问自己，NAT 回环不通时永远同步不上）。
|
||||
- 普通实例：默认参与共享，主站地址内置（SHARE_HUB_URL 可改成自建的私有社区）。每 30 分钟一轮：
|
||||
首次注册领取令牌 → 上报心跳（每 6 小时附带一次投票）→ 拉取社区库（没变化时主站回 304）→ 拉取部署量。
|
||||
社区库缓存在 DATA_DIR/share_community.json，主站暂时不可达时继续用上次的版本。
|
||||
|
||||
隐私与防污染：
|
||||
- 投票内容只有 (远端 IP / 域名, 应用 ID, 字节票数)；不含本机终端、域名记录、自定义应用；
|
||||
- 私网 / 保留 / 组播地址剔除；IPv6 按 /64 前缀聚合（中国环境以 IPv6 公网为主）；
|
||||
- 应用白名单：只共享所有实例都认识的应用，自定义应用和编造的 ID 拒收；
|
||||
- 单文件限量，防止单实例撑爆共享文件夹。
|
||||
隐私：
|
||||
- 投票只有 (远端 IP / 域名, 应用 ID, 字节票数)；不含本机终端、访问记录、自定义应用；
|
||||
- 私网 / 保留 / 组播地址剔除；IPv6 按 /64 前缀聚合；
|
||||
- 主站不保存来源 IP，只保存来源网络（IPv4 /24、IPv6 /56）的加盐哈希。
|
||||
|
||||
安全：WebDAV 地址建议用 HTTPS 或仅内网可达；账号密码保存在本实例数据库里，不上报。
|
||||
防污染：
|
||||
- 每个实例有主站发的令牌，别人不能冒充或覆盖它的投票；同一来源网络每天限注册次数；投票每小时最多收一次；
|
||||
- 一条映射要有 ≥2 个**不同来源网络**的实例投给同一个应用，并且票数多于其它应用，才进社区库——
|
||||
一个人伪造再多实例 ID，来自同一个网络也只算一票；
|
||||
- 应用白名单（自定义应用、编造的 ID 拒收）、域名格式校验、单实例条数上限、字节票数上限。
|
||||
"""
|
||||
import gzip
|
||||
import hashlib
|
||||
import hmac
|
||||
import ipaddress
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import secrets
|
||||
import threading
|
||||
import time
|
||||
import urllib.parse
|
||||
import uuid
|
||||
from collections import defaultdict
|
||||
|
||||
import httpx
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy import delete, func, select
|
||||
|
||||
from app.classify.local import _is_ip, _v6key
|
||||
from app.models import HostDomainDay
|
||||
from app.config import get_settings
|
||||
from app.models import HostDomainDay, Setting, ShareDeployment, ShareDomain, ShareEntry
|
||||
|
||||
MAX_UPLOAD = 50_000 # 单个实例投票文件的条数上限
|
||||
MAX_COMMUNITY = 200_000 # 社区聚合后的条数上限
|
||||
PUBLISH_FILES = 2 # 至少几个不同实例的投票文件见过才进社区层
|
||||
DEPLOY_ONLINE = 7200 # 心跳 2 小时内算在线
|
||||
API = "/api/share/v1"
|
||||
MAX_UPLOAD = 50_000 # 单次上报 IP / 域名各自的条数上限
|
||||
MAX_COMMUNITY = 200_000 # 社区库 IP / 域名各自的条数上限
|
||||
PUBLISH_NETS = 2 # 至少几个不同来源网络投给同一应用才发布
|
||||
DEPLOY_ONLINE = 7200 # 2 小时内有心跳算在线
|
||||
DEPLOY_KEEP = 90 * 86400 # 90 天没心跳的实例不再计入部署量
|
||||
ENTRY_TTL = 30 * 86400 # 30 天没再上报投票的实例，投票作废
|
||||
VOTES_EVERY = 6 * 3600 # 客户端每 6 小时附带一次投票（心跳每轮都发）
|
||||
VOTES_MIN_GAP = 3600 # 主站对同一实例的投票每小时最多收一次
|
||||
REGISTER_PER_NET = 5 # 同一来源网络每天最多注册几个实例
|
||||
MAX_BODY = 8 * 1024 * 1024 # 上报请求体上限（压缩后）
|
||||
MAX_INFLATED = 48 * 1024 * 1024 # 解压后上限，防压缩炸弹
|
||||
LIB_REBUILD_GAP = 60 # 社区库最快每分钟重建一次
|
||||
MAX_BYTES = 10 ** 15 # 单条字节票数上限
|
||||
HUB_NET = "hub" # 主站自己的投票的来源网络标记
|
||||
|
||||
# 域名格式（debug20：格式错误的域名条目直接丢弃）
|
||||
# 域名格式：格式错误的域名条目直接丢弃
|
||||
_DOMAIN_RE = re.compile(r"^(?=.{1,253}$)([a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?\.)+[a-z]{2,63}$")
|
||||
MAX_BYTES = 10 ** 15
|
||||
|
||||
|
||||
class ShareError(Exception):
|
||||
def __init__(self, status: int, detail: str):
|
||||
super().__init__(detail)
|
||||
self.status, self.detail = status, detail
|
||||
|
||||
|
||||
# ---------- 共用 ----------
|
||||
|
||||
_sync_lock = threading.Lock()
|
||||
_sync = {"ts": 0, "ok": None, "error": "", "community_ips": 0, "community_domains": 0,
|
||||
"deployed": 0, "online": 0, "offline": 0}
|
||||
"total": None, "online": None, "offline": None}
|
||||
_client = {"votes_ts": 0, "etag": ""}
|
||||
|
||||
|
||||
def sync_state() -> dict:
|
||||
@@ -50,19 +78,38 @@ def sync_state() -> dict:
|
||||
return dict(_sync)
|
||||
|
||||
|
||||
def instance_id(db) -> str:
|
||||
"""本实例的匿名 ID：第一次生成后存在 settings 表（不上报任何可识别信息）"""
|
||||
from app.models import Setting
|
||||
row = db.get(Setting, "share_instance_id")
|
||||
if row is not None and row.value:
|
||||
return row.value
|
||||
iid = str(uuid.uuid4())
|
||||
row = db.get(Setting, "share_instance_id")
|
||||
def _set_sync(**kw):
|
||||
with _sync_lock:
|
||||
_sync.update(kw)
|
||||
|
||||
|
||||
def is_hub() -> bool:
|
||||
return bool(get_settings().share_hub)
|
||||
|
||||
|
||||
def hub_url() -> str:
|
||||
return (get_settings().share_hub_url or "").strip().rstrip("/")
|
||||
|
||||
|
||||
def sharing_on(db) -> bool:
|
||||
"""共享识别库的总开关（debug16 第 3 条）：默认打开；关闭后不心跳、不上报、不拉取（主站不受影响，照常服务其它实例）"""
|
||||
row = db.get(Setting, "share_enabled")
|
||||
if row is not None and row.value is not None:
|
||||
return row.value == "1"
|
||||
return bool(get_settings().share_enabled)
|
||||
|
||||
|
||||
def _setting(db, key: str) -> str:
|
||||
row = db.get(Setting, key)
|
||||
return row.value if row is not None and row.value else ""
|
||||
|
||||
|
||||
def _put_setting(db, key: str, value: str):
|
||||
row = db.get(Setting, key)
|
||||
if row is None:
|
||||
db.add(Setting(key="share_instance_id", value=iid))
|
||||
db.add(Setting(key=key, value=value))
|
||||
else:
|
||||
row.value = iid
|
||||
return iid
|
||||
row.value = value
|
||||
|
||||
|
||||
def _is_shareable_app(classifier, app: str) -> bool:
|
||||
@@ -80,166 +127,446 @@ def _shareable_ip(ip: str) -> bool:
|
||||
return not (a.is_private or a.is_reserved or a.is_multicast or a.is_loopback or a.is_link_local)
|
||||
|
||||
|
||||
def export_entries(engine, db) -> list[dict]:
|
||||
"""本地自学习库的 IP 条目：host_domain_day 里以 IP 记录的行，按字节数取最多的应用。
|
||||
IPv4 按精确地址；IPv6 按 /64 前缀聚合（中国环境以 IPv6 公网为主，精确地址不稳定）"""
|
||||
def export_entries(engine, db) -> tuple[list[dict], list[dict]]:
|
||||
"""本地自学习库整理成投票：host_domain_day 里的行按字节数取最多的应用。
|
||||
IP 条目 IPv4 按精确地址、IPv6 按 /64 前缀（精确地址在不同网络间几乎不重复，凑不满投票）；域名条目按精确域名"""
|
||||
classifier = engine.classifier
|
||||
votes: dict = defaultdict(dict)
|
||||
rows = db.execute(select(HostDomainDay.domain, HostDomainDay.app, HostDomainDay.bytes)).all()
|
||||
for dom, app, b in rows:
|
||||
if not _shareable_ip(dom) or not _is_shareable_app(classifier, app):
|
||||
continue
|
||||
key = _v6key(dom) or dom
|
||||
per = votes[key]
|
||||
per[app] = per.get(app, 0) + int(b or 0)
|
||||
out = [{"ip": ip, "app": max(per, key=per.get), "bytes": sum(per.values())}
|
||||
for ip, per in votes.items()]
|
||||
out.sort(key=lambda x: -x["bytes"])
|
||||
doms = export_domains(engine, db)
|
||||
return out[:MAX_UPLOAD], doms
|
||||
|
||||
|
||||
def export_domains(engine, db) -> list[dict]:
|
||||
"""本地自学习库的域名条目（debug17）：非 IP 的域名行，按字节数取最多的应用，精确域名共享"""
|
||||
classifier = engine.classifier
|
||||
votes: dict = defaultdict(dict)
|
||||
rows = db.execute(select(HostDomainDay.domain, HostDomainDay.app, HostDomainDay.bytes)).all()
|
||||
for dom, app, b in rows:
|
||||
if _is_ip(dom) or not _is_shareable_app(classifier, app):
|
||||
continue
|
||||
per = votes[dom]
|
||||
per[app] = per.get(app, 0) + int(b or 0)
|
||||
out = [{"domain": d, "app": max(p, key=p.get), "bytes": sum(p.values())}
|
||||
for d, p in votes.items()]
|
||||
out.sort(key=lambda x: -x["bytes"])
|
||||
return out[:MAX_UPLOAD]
|
||||
|
||||
|
||||
# ---------- WebDAV ----------
|
||||
|
||||
def _dav_url(base: str, path: str) -> str:
|
||||
return base.rstrip("/") + "/" + "/".join(urllib.parse.quote(x) for x in path.split("/"))
|
||||
|
||||
|
||||
def _dav_put(httpx, base: str, path: str, data, auth) -> bool:
|
||||
"""PUT 文件；目标文件夹不存在（409）时逐级 MKCOL 后重试一次"""
|
||||
url = _dav_url(base, path)
|
||||
r = httpx.put(url, content=data, auth=auth, timeout=30.0, verify=False)
|
||||
if r.status_code in (200, 201, 204):
|
||||
return True
|
||||
if r.status_code == 409: # 上级文件夹不存在：逐级创建
|
||||
parts = path.split("/")
|
||||
for i in range(1, len(parts)):
|
||||
httpx.request("MKCOL", _dav_url(base, "/".join(parts[:i])), auth=auth,
|
||||
timeout=30.0, verify=False)
|
||||
r = httpx.put(url, content=data, auth=auth, timeout=30.0, verify=False)
|
||||
return r.status_code in (200, 201, 204)
|
||||
return False
|
||||
|
||||
|
||||
def _dav_list(httpx, base: str, folder: str, auth) -> list[str]:
|
||||
"""PROPFIND 列出文件夹里的 .json 文件，返回 folder/name 相对路径（和 _dav_get 拼接一致）"""
|
||||
r = httpx.request("PROPFIND", _dav_url(base, folder), headers={"Depth": "1"}, auth=auth,
|
||||
timeout=30.0, verify=False)
|
||||
if r.status_code not in (200, 207):
|
||||
return []
|
||||
hrefs = re.findall(r"<(?:[\w]+:)?href>\s*([^<]+)\s*</(?:[\w]+:)?href>", r.text, re.IGNORECASE)
|
||||
base_path = urllib.parse.quote(_dav_url(base, "").split("//", 1)[-1].split("/", 1)[-1])
|
||||
out = []
|
||||
for h in hrefs:
|
||||
name = urllib.parse.unquote(h.split("?")[0].rstrip("/")).rsplit("/", 1)[-1]
|
||||
if name.endswith(".json") and name != folder:
|
||||
out.append(f"{folder}/{name}")
|
||||
return out
|
||||
|
||||
|
||||
def _dav_get(httpx, base: str, path: str, auth):
|
||||
r = httpx.get(_dav_url(base, path), auth=auth, timeout=30.0, verify=False)
|
||||
if r.status_code != 200:
|
||||
return None
|
||||
try:
|
||||
return json.loads(r.text)
|
||||
except Exception: # noqa: BLE001
|
||||
return None
|
||||
|
||||
|
||||
def webdav_cycle(engine, db, base: str, user: str, password: str) -> dict:
|
||||
"""一轮 WebDAV 共享（每 30 分钟）：写自己的心跳与投票文件 → 读其它实例的文件 → 本地聚合"""
|
||||
from app import __version__
|
||||
auth = (user, password) if user or password else None
|
||||
iid = instance_id(db)
|
||||
now = int(time.time())
|
||||
|
||||
# 1) 心跳（部署量统计：只有版本号和时间）
|
||||
ok_put_dep = _dav_put(httpx, base, f"deployments/{iid}.json",
|
||||
json.dumps({"version": __version__, "ts": now}).encode(), auth)
|
||||
# 2) 投票（本网络实测）
|
||||
entries, domains = export_entries(engine, db)
|
||||
ok_put_votes = _dav_put(httpx, base, f"votes/{iid}.json",
|
||||
json.dumps({"version": __version__, "ts": now,
|
||||
"entries": entries, "domains": domains},
|
||||
ensure_ascii=False).encode(), auth)
|
||||
|
||||
# 3) 读其它实例的文件，本地聚合（自己写不进去时留给下轮，不空算）
|
||||
other_dep, other_votes = [], []
|
||||
if ok_put_dep and ok_put_votes:
|
||||
for kind, out in (("deployments", other_dep), ("votes", other_votes)):
|
||||
for path in _dav_list(httpx, base, kind, auth):
|
||||
if path == f"{kind}/{iid}.json":
|
||||
continue
|
||||
data = _dav_get(httpx, base, path, auth)
|
||||
if isinstance(data, dict):
|
||||
out.append(data)
|
||||
|
||||
# 部署量：心跳文件数 = 已部署；ts 在 2 小时内 = 在线
|
||||
deployed, online = len(other_dep) + (1 if ok_put_dep else 0), 0
|
||||
for d in other_dep:
|
||||
if int(d.get("ts", 0) or 0) >= now - DEPLOY_ONLINE:
|
||||
online += 1
|
||||
if ok_put_dep:
|
||||
online += 1
|
||||
|
||||
# 社区层：出现在 ≥2 个不同投票文件里的映射（自己的文件算一票）。votes[k] = {app: (实例数, 总字节)}
|
||||
def _vote(bucket, key, app, weight):
|
||||
per = bucket[key]
|
||||
n, b = per.get(app, (0, 0))
|
||||
per[app] = (n + 1, b + weight)
|
||||
|
||||
ip_votes: dict = defaultdict(dict)
|
||||
dom_votes: dict = defaultdict(dict)
|
||||
if ok_put_votes:
|
||||
for e in entries:
|
||||
_vote(ip_votes, e["ip"], e["app"], e["bytes"])
|
||||
for e in domains:
|
||||
_vote(dom_votes, e["domain"], e["app"], e["bytes"])
|
||||
for d in other_votes:
|
||||
for e in d.get("entries") or []:
|
||||
ip, app, b = str(e.get("ip", "")), str(e.get("app", "")), int(e.get("bytes", 0) or 0)
|
||||
if _shareable_ip(ip):
|
||||
_vote(ip_votes, ip, app, b)
|
||||
for e in d.get("domains") or []:
|
||||
dom_, app, b = str(e.get("domain", "")).lower(), str(e.get("app", "")), int(e.get("bytes", 0) or 0)
|
||||
if dom_ and not _is_ip(dom_):
|
||||
_vote(dom_votes, dom_, app, b)
|
||||
rows = db.execute(select(HostDomainDay.domain, HostDomainDay.app, HostDomainDay.bytes)).all()
|
||||
for dom, app, b in rows:
|
||||
if not dom or not _is_shareable_app(classifier, app):
|
||||
continue
|
||||
if _is_ip(dom):
|
||||
if not _shareable_ip(dom):
|
||||
continue
|
||||
per = ip_votes[_v6key(dom) or dom]
|
||||
else:
|
||||
per = dom_votes[dom.lower().rstrip(".")]
|
||||
per[app] = per.get(app, 0) + int(b or 0)
|
||||
|
||||
def top(bucket, cap):
|
||||
out = {}
|
||||
for key, per in sorted(bucket.items(), key=lambda kv: -sum(v[1] for v in kv[1].values()))[:cap]:
|
||||
if any(n >= PUBLISH_FILES for n, _b in per.values()): # 至少 2 个实例见过
|
||||
out[key] = max(per, key=lambda a: per[a][1])
|
||||
return out
|
||||
def pack(votes, field):
|
||||
out = [{field: k, "app": max(p, key=p.get), "bytes": sum(p.values())} for k, p in votes.items()]
|
||||
out.sort(key=lambda x: -x["bytes"])
|
||||
return out[:MAX_UPLOAD]
|
||||
return pack(ip_votes, "ip"), pack(dom_votes, "domain")
|
||||
|
||||
community_ips = top(ip_votes, MAX_COMMUNITY)
|
||||
community_domains = top(dom_votes, MAX_COMMUNITY)
|
||||
|
||||
def apply_library(data: dict) -> tuple[int, int]:
|
||||
"""把社区库并入本地自学习库的社区层（本地实测优先，社区只补空白）。返回 (IP 条数, 域名条数)"""
|
||||
from app.classify.local import local
|
||||
local.load_community(community_ips, community_domains)
|
||||
ips, doms = {}, {}
|
||||
for e in (data or {}).get("entries") or []:
|
||||
if not isinstance(e, dict):
|
||||
continue
|
||||
ip, app = str(e.get("ip", "")), str(e.get("app", ""))
|
||||
if app and len(app) <= 48 and _shareable_ip(ip[:-3] if ip.endswith("/64") else ip):
|
||||
ips[ip] = app
|
||||
for e in (data or {}).get("domains") or []:
|
||||
if not isinstance(e, dict):
|
||||
continue
|
||||
dom, app = str(e.get("domain", "")).lower().rstrip("."), str(e.get("app", ""))
|
||||
if app and len(app) <= 48 and _DOMAIN_RE.match(dom):
|
||||
doms[dom] = app
|
||||
local.load_community(ips, doms)
|
||||
return len(ips), len(doms)
|
||||
|
||||
ok = ok_put_dep and ok_put_votes
|
||||
error = "" if ok else "WebDAV 写入失败：检查地址 / 账号密码 / 文件夹权限"
|
||||
with _sync_lock:
|
||||
_sync.update(ts=now, ok=ok, error=error, deployed=deployed, online=online,
|
||||
offline=deployed - online,
|
||||
community_ips=len(community_ips), community_domains=len(community_domains))
|
||||
return {"deployed": deployed, "online": online, "offline": deployed - online,
|
||||
"community_ips": len(community_ips), "community_domains": len(community_domains),
|
||||
"uploaded": bool(ok_put_dep and ok_put_votes)}
|
||||
|
||||
def _cache_path() -> str:
|
||||
return os.path.join(os.environ.get("DATA_DIR", "data"), "share_community.json")
|
||||
|
||||
|
||||
def load_cached_library():
|
||||
"""启动时先用上次拉到的社区库，不用等第一轮同步；主站暂时不可达时也一直有数据"""
|
||||
try:
|
||||
with open(_cache_path(), encoding="utf-8") as f:
|
||||
data = json.load(f)
|
||||
except (OSError, ValueError):
|
||||
return
|
||||
n_ip, n_dom = apply_library(data)
|
||||
_client["etag"] = str(data.get("etag") or "")
|
||||
_set_sync(community_ips=n_ip, community_domains=n_dom)
|
||||
|
||||
|
||||
def _save_cache(data: dict, etag: str):
|
||||
try:
|
||||
os.makedirs(os.path.dirname(_cache_path()) or ".", exist_ok=True)
|
||||
tmp = _cache_path() + ".tmp"
|
||||
with open(tmp, "w", encoding="utf-8") as f:
|
||||
json.dump({**data, "etag": etag}, f, ensure_ascii=False)
|
||||
os.replace(tmp, _cache_path())
|
||||
except OSError as e:
|
||||
print(f"[share] 社区库缓存写入失败：{e}", flush=True)
|
||||
|
||||
|
||||
# ---------- 社区主站 ----------
|
||||
|
||||
def net_of(db, ip: str) -> str:
|
||||
"""来源网络的加盐哈希（IPv4 /24、IPv6 /56）：同一网络里伪造多个实例也只算一票；主站不保存来源 IP 本身。
|
||||
IPv6 按 /56：国内家庭宽带一般分到 /56 或 /60，按 /48 会把几百户人家算成同一个网络"""
|
||||
try:
|
||||
a = ipaddress.ip_address(ip)
|
||||
except ValueError:
|
||||
return "unknown"
|
||||
net = ipaddress.ip_network(f"{a}/{24 if a.version == 4 else 56}", strict=False)
|
||||
salt = _setting(db, "share_salt")
|
||||
if not salt:
|
||||
salt = secrets.token_hex(16)
|
||||
_put_setting(db, "share_salt", salt)
|
||||
return hashlib.sha256(f"{salt}|{net}".encode()).hexdigest()[:16]
|
||||
|
||||
|
||||
def client_ip(peer: str, forwarded: str | None, real_ip: str | None) -> str:
|
||||
"""主站看到的来源地址。直接暴露在公网时用 TCP 对端；对端是内网（前面有反向代理）时，取代理追加在
|
||||
X-Forwarded-For 最右边的公网地址——直连的人伪造这个头没用，因为那时对端是公网地址、头会被忽略"""
|
||||
try:
|
||||
p = ipaddress.ip_address(peer)
|
||||
behind_proxy = p.is_private or p.is_loopback
|
||||
except ValueError:
|
||||
behind_proxy = False
|
||||
if not behind_proxy:
|
||||
return peer
|
||||
for cand in reversed([x.strip() for x in (forwarded or "").split(",") if x.strip()]):
|
||||
if _shareable_ip(cand):
|
||||
return cand
|
||||
if real_ip and _shareable_ip(real_ip.strip()):
|
||||
return real_ip.strip()
|
||||
return peer
|
||||
|
||||
|
||||
def _hash_token(token: str) -> str:
|
||||
return hashlib.sha256(token.encode()).hexdigest()
|
||||
|
||||
|
||||
def register(db, net: str, version: str) -> tuple[str, str]:
|
||||
"""主站：登记一个新实例并发放令牌（调用方提交事务）。同一来源网络每天限 REGISTER_PER_NET 个"""
|
||||
now = int(time.time())
|
||||
recent = db.scalar(select(func.count()).select_from(ShareDeployment)
|
||||
.where(ShareDeployment.net == net, ShareDeployment.first_seen >= now - 86400)) or 0
|
||||
if net != HUB_NET and recent >= REGISTER_PER_NET:
|
||||
raise ShareError(429, "注册太频繁，请明天再试")
|
||||
iid, token = str(uuid.uuid4()), secrets.token_urlsafe(32)
|
||||
db.add(ShareDeployment(instance=iid, version=version[:16], first_seen=now, last_seen=now,
|
||||
token_hash=_hash_token(token), net=net))
|
||||
return iid, token
|
||||
|
||||
|
||||
def authenticate(db, auth_header: str | None) -> ShareDeployment:
|
||||
"""主站：校验 Authorization: Bearer <实例ID>:<令牌>"""
|
||||
raw = (auth_header or "").removeprefix("Bearer ").strip()
|
||||
iid, _, token = raw.partition(":")
|
||||
row = db.get(ShareDeployment, iid[:36]) if iid and token else None
|
||||
if row is None or not row.token_hash or not hmac.compare_digest(row.token_hash, _hash_token(token)):
|
||||
raise ShareError(401, "实例未注册或令牌无效")
|
||||
return row
|
||||
|
||||
|
||||
def _clamp_bytes(v) -> int:
|
||||
try:
|
||||
return min(max(int(v or 0), 0), MAX_BYTES)
|
||||
except (TypeError, ValueError):
|
||||
return 0
|
||||
|
||||
|
||||
def report(db, row: ShareDeployment, net: str, body: dict, classifier) -> dict:
|
||||
"""主站：记录一次上报（调用方提交事务）。心跳每次都记；带了投票时整份替换该实例的旧投票。
|
||||
应用白名单：只收主站也认识的内置应用，自定义应用和编造的 ID 拒收"""
|
||||
now = int(time.time())
|
||||
row.last_seen, row.net = now, net
|
||||
row.version = str(body.get("version") or "")[:16]
|
||||
entries, domains = body.get("entries"), body.get("domains")
|
||||
if not isinstance(entries, list) and not isinstance(domains, list):
|
||||
return {"ok": True, "accepted": 0}
|
||||
if row.last_votes and now - row.last_votes < VOTES_MIN_GAP:
|
||||
return {"ok": True, "accepted": 0, "skipped": "投票每小时最多收一次"}
|
||||
ips: dict = {}
|
||||
for e in (entries or [])[:MAX_UPLOAD]:
|
||||
if not isinstance(e, dict):
|
||||
continue
|
||||
ip, app = str(e.get("ip", "")), str(e.get("app", ""))
|
||||
base = ip[:-3] if ip.endswith("/64") else ip
|
||||
if not _shareable_ip(base) or not _is_shareable_app(classifier, app):
|
||||
continue
|
||||
ips[_v6key(base) or base] = (app, _clamp_bytes(e.get("bytes")))
|
||||
doms: dict = {}
|
||||
for e in (domains or [])[:MAX_UPLOAD]:
|
||||
if not isinstance(e, dict):
|
||||
continue
|
||||
dom, app = str(e.get("domain", "")).lower().rstrip("."), str(e.get("app", ""))
|
||||
if not _DOMAIN_RE.match(dom) or not _is_shareable_app(classifier, app):
|
||||
continue
|
||||
doms[dom] = (app, _clamp_bytes(e.get("bytes")))
|
||||
db.execute(delete(ShareEntry).where(ShareEntry.reporter == row.instance))
|
||||
db.execute(delete(ShareDomain).where(ShareDomain.reporter == row.instance))
|
||||
db.add_all([ShareEntry(ip=k, app=a, reporter=row.instance, bytes=b, ts=now) for k, (a, b) in ips.items()])
|
||||
db.add_all([ShareDomain(domain=k, app=a, reporter=row.instance, bytes=b, ts=now) for k, (a, b) in doms.items()])
|
||||
row.last_votes = now
|
||||
_lib["dirty"] = True
|
||||
return {"ok": True, "accepted": len(ips) + len(doms)}
|
||||
|
||||
|
||||
def purge(db):
|
||||
"""主站：清掉 90 天没心跳的实例、30 天没更新的投票（调用方提交事务）"""
|
||||
now = int(time.time())
|
||||
db.execute(delete(ShareDeployment).where(ShareDeployment.last_seen < now - DEPLOY_KEEP))
|
||||
n = db.execute(delete(ShareEntry).where(ShareEntry.ts < now - ENTRY_TTL)).rowcount or 0
|
||||
n += db.execute(delete(ShareDomain).where(ShareDomain.ts < now - ENTRY_TTL)).rowcount or 0
|
||||
if n:
|
||||
_lib["dirty"] = True
|
||||
|
||||
|
||||
def _aggregate(rows, nets: dict) -> dict:
|
||||
"""按来源网络计票：每个键选投票网络最多的应用（平票看字节），网络数 ≥ PUBLISH_NETS 且多于第二名才发布"""
|
||||
votes: dict = defaultdict(lambda: defaultdict(lambda: [set(), 0]))
|
||||
for key, app, reporter, b in rows:
|
||||
net = nets.get(reporter)
|
||||
if net is None: # 已被清理的实例、0.6.20 以前的旧中转留下的投票不算
|
||||
continue
|
||||
v = votes[key][app]
|
||||
v[0].add(net)
|
||||
v[1] += int(b or 0)
|
||||
out = []
|
||||
for key, per in votes.items():
|
||||
ranked = sorted(per.items(), key=lambda kv: (len(kv[1][0]), kv[1][1]), reverse=True)
|
||||
top_app, (top_nets, top_bytes) = ranked[0]
|
||||
runner = len(ranked[1][1][0]) if len(ranked) > 1 else 0
|
||||
if len(top_nets) >= PUBLISH_NETS and len(top_nets) > runner:
|
||||
out.append((top_bytes, key, top_app))
|
||||
out.sort(reverse=True)
|
||||
return {key: app for _b, key, app in out[:MAX_COMMUNITY]}
|
||||
|
||||
|
||||
def build_library(db) -> dict:
|
||||
nets = {i: n for i, n in db.execute(select(ShareDeployment.instance, ShareDeployment.net)).all() if n}
|
||||
ips = _aggregate(db.execute(select(ShareEntry.ip, ShareEntry.app, ShareEntry.reporter, ShareEntry.bytes)).all(), nets)
|
||||
doms = _aggregate(db.execute(select(ShareDomain.domain, ShareDomain.app, ShareDomain.reporter,
|
||||
ShareDomain.bytes)).all(), nets)
|
||||
return {"version": 1, "generated": int(time.time()),
|
||||
"entries": [{"ip": k, "app": v} for k, v in sorted(ips.items())],
|
||||
"domains": [{"domain": k, "app": v} for k, v in sorted(doms.items())]}
|
||||
|
||||
|
||||
_lib_lock = threading.Lock()
|
||||
_lib = {"dirty": True, "ts": 0, "etag": "", "json": b"", "gzip": b"", "data": None}
|
||||
|
||||
|
||||
def library(db) -> dict:
|
||||
"""主站：当前社区库（有新投票时最快每分钟重建一次），返回 {etag, json, gzip, data}"""
|
||||
with _lib_lock:
|
||||
if _lib["data"] is None or (_lib["dirty"] and time.time() - _lib["ts"] >= LIB_REBUILD_GAP):
|
||||
data = build_library(db)
|
||||
raw = json.dumps({k: data[k] for k in ("version", "entries", "domains")}, ensure_ascii=False,
|
||||
separators=(",", ":")).encode()
|
||||
body = json.dumps(data, ensure_ascii=False, separators=(",", ":")).encode()
|
||||
_lib.update(dirty=False, ts=time.time(), etag='"' + hashlib.sha256(raw).hexdigest()[:32] + '"',
|
||||
json=body, gzip=gzip.compress(body, 6), data=data)
|
||||
return dict(_lib)
|
||||
|
||||
|
||||
def reset_library_cache():
|
||||
"""测试用"""
|
||||
with _lib_lock:
|
||||
_lib.update(dirty=True, ts=0, etag="", json=b"", gzip=b"", data=None)
|
||||
_client.update(votes_ts=0, etag="")
|
||||
|
||||
|
||||
def stats(db) -> dict:
|
||||
"""全球部署量（公开）：已部署 = 90 天内有心跳的实例，在线 = 2 小时内有心跳"""
|
||||
now = int(time.time())
|
||||
# 只数 0.6.21 起注册过的实例：旧中转（0.6.10~0.6.19）留下的无令牌心跳不计，否则升级后会重复计数
|
||||
q = select(func.count()).select_from(ShareDeployment).where(ShareDeployment.token_hash.is_not(None))
|
||||
total = db.scalar(q.where(ShareDeployment.last_seen >= now - DEPLOY_KEEP)) or 0
|
||||
online = db.scalar(q.where(ShareDeployment.last_seen >= now - DEPLOY_ONLINE)) or 0
|
||||
return {"total": total, "online": online, "offline": total - online}
|
||||
|
||||
|
||||
def inflate(body: bytes, encoding: str | None) -> dict:
|
||||
"""主站：解析上报请求体（可 gzip 压缩），限制解压后大小"""
|
||||
if len(body) > MAX_BODY:
|
||||
raise ShareError(413, "上报内容太大")
|
||||
if (encoding or "").lower() == "gzip":
|
||||
import zlib
|
||||
d = zlib.decompressobj(16 + zlib.MAX_WBITS)
|
||||
try:
|
||||
out = d.decompress(body, MAX_INFLATED)
|
||||
except zlib.error:
|
||||
raise ShareError(400, "上报内容解压失败") from None
|
||||
if d.unconsumed_tail:
|
||||
raise ShareError(413, "上报内容太大")
|
||||
body = out
|
||||
try:
|
||||
data = json.loads(body or b"{}")
|
||||
except ValueError:
|
||||
raise ShareError(400, "上报内容不是 JSON") from None
|
||||
if not isinstance(data, dict):
|
||||
raise ShareError(400, "上报内容格式不对")
|
||||
return data
|
||||
|
||||
|
||||
def hub_cycle(engine, db) -> dict:
|
||||
"""主站自己的一轮：投票在进程内直接计入（不经网络）→ 清理过期 → 重建社区库并用到本机"""
|
||||
from app import __version__
|
||||
iid = _setting(db, "share_hub_instance")
|
||||
row = db.get(ShareDeployment, iid) if iid else None
|
||||
if row is None:
|
||||
iid, _token = register(db, HUB_NET, __version__)
|
||||
_put_setting(db, "share_hub_instance", iid)
|
||||
db.flush()
|
||||
row = db.get(ShareDeployment, iid)
|
||||
body: dict = {"version": __version__}
|
||||
on = sharing_on(db)
|
||||
if on and (not row.last_votes or time.time() - row.last_votes >= VOTES_EVERY):
|
||||
body["entries"], body["domains"] = export_entries(engine, db)
|
||||
elif not on:
|
||||
# 主站关掉“参与共享”：照常服务其它实例，只是不再贡献自己的投票、本机也不用社区库
|
||||
db.execute(delete(ShareEntry).where(ShareEntry.reporter == iid))
|
||||
db.execute(delete(ShareDomain).where(ShareDomain.reporter == iid))
|
||||
row.last_votes = None
|
||||
_lib["dirty"] = True
|
||||
report(db, row, HUB_NET, body, engine.classifier)
|
||||
purge(db)
|
||||
db.flush()
|
||||
lib = library(db)
|
||||
if on:
|
||||
n_ip, n_dom = apply_library(lib["data"])
|
||||
else:
|
||||
from app.classify.local import local
|
||||
local.load_community({}, {})
|
||||
n_ip = n_dom = 0
|
||||
st = stats(db)
|
||||
_set_sync(ts=int(time.time()), ok=True, error="", community_ips=n_ip, community_domains=n_dom, **st)
|
||||
return {**st, "community_ips": n_ip, "community_domains": n_dom}
|
||||
|
||||
|
||||
# ---------- 普通实例 ----------
|
||||
|
||||
def _http() -> httpx.Client:
|
||||
from app import __version__
|
||||
return httpx.Client(verify=get_settings().share_verify_tls, timeout=30.0, follow_redirects=False,
|
||||
headers={"User-Agent": f"ros-dpi/{__version__}"})
|
||||
|
||||
|
||||
def _err(e: Exception) -> str:
|
||||
if isinstance(e, httpx.HTTPStatusError):
|
||||
try:
|
||||
detail = e.response.json().get("detail") or ""
|
||||
except ValueError:
|
||||
detail = ""
|
||||
return f"主站返回 {e.response.status_code}" + (f"：{detail}" if detail else "")
|
||||
if isinstance(e, httpx.ConnectError) and "CERTIFICATE" in str(e).upper():
|
||||
return "主站证书校验失败（私有社区用自签证书时设 SHARE_VERIFY_TLS=false）"
|
||||
if isinstance(e, ShareError):
|
||||
return e.detail
|
||||
msg = str(e).lower()
|
||||
if isinstance(e, httpx.ConnectError) and ("errno 101" in msg or "unreachable" in msg
|
||||
or "cannot assign requested address" in msg):
|
||||
# 实例不需要公网 IP（都是主动连主站），但主站目前只有 IPv6 地址：本机 / 容器要有 IPv6 出口
|
||||
return "连不上社区主站：本机没有 IPv6 出口（主站只有 IPv6 地址）。Docker 部署请按 docker-compose.yml 末尾的说明开启 IPv6，或改用 Linux 一键安装"
|
||||
return f"{type(e).__name__}: {str(e)[:120]}"
|
||||
|
||||
|
||||
def _register(c: httpx.Client, base: str, db) -> tuple[str, str]:
|
||||
from app import __version__
|
||||
r = c.post(base + API + "/register", json={"version": __version__})
|
||||
r.raise_for_status()
|
||||
d = r.json()
|
||||
iid, token = str(d["instance"]), str(d["token"])
|
||||
_put_setting(db, "share_client_instance", iid)
|
||||
_put_setting(db, "share_client_token", token)
|
||||
_put_setting(db, "share_client_hub", base)
|
||||
db.commit()
|
||||
return iid, token
|
||||
|
||||
|
||||
def client_cycle(engine, db) -> dict:
|
||||
"""普通实例一轮同步：注册（首次）→ 心跳（每 6 小时带投票）→ 拉社区库 → 拉部署量"""
|
||||
from app import __version__
|
||||
base = hub_url()
|
||||
if not base:
|
||||
raise ShareError(400, "没有配置社区主站地址（SHARE_HUB_URL）")
|
||||
with _http() as c:
|
||||
iid, token = _setting(db, "share_client_instance"), _setting(db, "share_client_token")
|
||||
if not iid or not token or _setting(db, "share_client_hub") != base: # 换了主站要重新注册
|
||||
iid, token = _register(c, base, db)
|
||||
_client["votes_ts"] = 0
|
||||
body: dict = {"version": __version__}
|
||||
with_votes = time.time() - _client["votes_ts"] >= VOTES_EVERY
|
||||
if with_votes:
|
||||
body["entries"], body["domains"] = export_entries(engine, db)
|
||||
payload = gzip.compress(json.dumps(body, ensure_ascii=False, separators=(",", ":")).encode(), 6)
|
||||
|
||||
def send(i, t):
|
||||
return c.post(base + API + "/report", content=payload,
|
||||
headers={"Authorization": f"Bearer {i}:{t}", "Content-Type": "application/json",
|
||||
"Content-Encoding": "gzip"})
|
||||
r = send(iid, token)
|
||||
if r.status_code == 401: # 主站清理过（90 天没心跳）或换过库：重新注册一次
|
||||
iid, token = _register(c, base, db)
|
||||
r = send(iid, token)
|
||||
r.raise_for_status()
|
||||
if with_votes:
|
||||
_client["votes_ts"] = time.time()
|
||||
|
||||
headers = {"If-None-Match": _client["etag"]} if _client["etag"] else {}
|
||||
r = c.get(base + API + "/library", headers=headers)
|
||||
if r.status_code == 304:
|
||||
st = sync_state()
|
||||
n_ip, n_dom = st["community_ips"], st["community_domains"]
|
||||
else:
|
||||
r.raise_for_status()
|
||||
data = r.json()
|
||||
n_ip, n_dom = apply_library(data)
|
||||
_client["etag"] = r.headers.get("etag", "")
|
||||
_save_cache(data, _client["etag"])
|
||||
r = c.get(base + API + "/stats")
|
||||
r.raise_for_status()
|
||||
st = {k: int(r.json().get(k) or 0) for k in ("total", "online", "offline")}
|
||||
_set_sync(ts=int(time.time()), ok=True, error="", community_ips=n_ip, community_domains=n_dom, **st)
|
||||
return {**st, "community_ips": n_ip, "community_domains": n_dom}
|
||||
|
||||
|
||||
def cycle(engine, db) -> dict | None:
|
||||
"""定时任务入口（每 30 分钟）：主站走进程内流程；普通实例开关关闭时什么都不发"""
|
||||
if is_hub():
|
||||
return hub_cycle(engine, db)
|
||||
if not sharing_on(db):
|
||||
from app.classify.local import local
|
||||
local.load_community({}, {})
|
||||
_set_sync(ts=0, ok=None, error="", community_ips=0, community_domains=0,
|
||||
total=None, online=None, offline=None)
|
||||
return None
|
||||
try:
|
||||
return client_cycle(engine, db)
|
||||
except Exception as e: # noqa: BLE001 记录原因给设置页显示；社区库继续用上次缓存
|
||||
_set_sync(ts=int(time.time()), ok=False, error=_err(e))
|
||||
raise
|
||||
|
||||
|
||||
def kick(fn):
|
||||
"""后台立刻跑一轮（启动后、打开开关后），不等 30 分钟的定时任务；测试环境不跑"""
|
||||
if not os.environ.get("ROSDPI_NO_JOBS"):
|
||||
threading.Thread(target=fn, daemon=True).start()
|
||||
|
||||
|
||||
def run_once():
|
||||
from app import database
|
||||
from app.engine import engine
|
||||
try:
|
||||
with database.session() as db:
|
||||
cycle(engine, db)
|
||||
db.commit()
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f"[share] 同步失败：{_err(e)}", flush=True)
|
||||
|
||||
|
||||
def status(db) -> dict:
|
||||
"""设置页的共享状态：角色、主站地址、开关、上次同步结果与全球部署量"""
|
||||
st = sync_state()
|
||||
if is_hub(): # 主站的部署量直接查库，不用等下一轮
|
||||
st.update(stats(db))
|
||||
# 不同来源网络数：实例很多却只有 1 个网络，说明反向代理没有传 X-Forwarded-For，社区库永远凑不够票
|
||||
st["nets"] = db.scalar(select(func.count(func.distinct(ShareDeployment.net)))
|
||||
.where(ShareDeployment.token_hash.is_not(None),
|
||||
ShareDeployment.net != HUB_NET)) or 0
|
||||
return {"role": "hub" if is_hub() else "client", "hub_url": hub_url(), "enabled": sharing_on(db),
|
||||
"verify_tls": bool(get_settings().share_verify_tls), "sync": st}
|
||||
+227
-125
@@ -1,152 +1,254 @@
|
||||
"""debug16/22：WebDAV 社区共享——本地库导出过滤、≥2 实例聚合、部署量、自学习接入。"""
|
||||
"""社区共享库（debug16 → 0.6.21 重做）：项目自带社区主站——注册 / 令牌、心跳 + 投票、按来源网络聚合防刷票、
|
||||
ETag 304、部署量；普通实例的完整同步流程走真实的主站端点（TestClient），不打桩协议本身。"""
|
||||
import gzip
|
||||
import json
|
||||
import time
|
||||
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from app import database, share
|
||||
from app.classify.local import local
|
||||
from app.engine import Engine
|
||||
from app.models import HostDomainDay
|
||||
from app.share import webdav_cycle
|
||||
from app.config import get_settings
|
||||
from app.engine import engine
|
||||
from app.main import app
|
||||
from app.models import HostDomainDay, Setting, ShareDeployment, ShareDomain, ShareEntry
|
||||
from app.routers import share as share_router
|
||||
|
||||
DAY = 1792900000
|
||||
HOST = "IP:192.168.8.40"
|
||||
BASE = "http://nas.local:5005/ros-dpi"
|
||||
HOST = "IP:192.168.88.41"
|
||||
HUB = "http://testserver"
|
||||
|
||||
|
||||
def _engine():
|
||||
e = Engine()
|
||||
e.set_lan("192.168.8.0/24")
|
||||
return e
|
||||
def _clean(db):
|
||||
for m in (ShareEntry, ShareDomain, ShareDeployment):
|
||||
db.query(m).delete(synchronize_session=False)
|
||||
db.query(HostDomainDay).filter(HostDomainDay.host == HOST).delete(synchronize_session=False)
|
||||
for k in ("share_client_instance", "share_client_token", "share_client_hub", "share_hub_instance"):
|
||||
row = db.get(Setting, k)
|
||||
if row is not None:
|
||||
db.delete(row)
|
||||
db.commit()
|
||||
|
||||
|
||||
class _FakeDav:
|
||||
"""最小 WebDAV 仿真：按文件夹分桶的内存键值 + PROPFIND 列表；记录所有 PUT。
|
||||
files 的键是 "<folder>/<name>.json"（和真实 WebDAV 的路径一致）"""
|
||||
@pytest.fixture()
|
||||
def hub(monkeypatch, tmp_path):
|
||||
"""本进程同时当社区主站和普通实例：主站端点由 TestClient 提供；X-Test-IP 头模拟不同实例的来源地址"""
|
||||
s = get_settings()
|
||||
monkeypatch.setattr(s, "share_hub", True)
|
||||
monkeypatch.setattr(s, "share_hub_url", HUB)
|
||||
monkeypatch.setattr(share, "LIB_REBUILD_GAP", 0)
|
||||
monkeypatch.setenv("DATA_DIR", str(tmp_path))
|
||||
monkeypatch.setattr(share_router, "_source", lambda req: req.headers.get("x-test-ip") or "203.0.113.9")
|
||||
database.connect_from_saved(force=True)
|
||||
with database.session() as db:
|
||||
_clean(db)
|
||||
share.reset_library_cache()
|
||||
local.reset()
|
||||
c = TestClient(app)
|
||||
state = {"ip": "198.51.100.1"}
|
||||
|
||||
def __init__(self):
|
||||
self.files = {}
|
||||
self.puts = []
|
||||
class _Client:
|
||||
"""client_cycle 用的 httpx.Client 替身：请求转给 TestClient，并带上当前模拟的来源地址"""
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def handle(self, method, url, content=None):
|
||||
from httpx import Response
|
||||
path = str(url).split("/ros-dpi/")[-1].rstrip("/") # deployments/xxx.json 或 votes/ 或 votes/xxx.json
|
||||
if method == "PROPFIND":
|
||||
folder = path.rstrip("/")
|
||||
names = [k for k in self.files if k.startswith(folder + "/")]
|
||||
body = "".join(f"<d:response><d:href>/{n}</d:href></d:response>" for n in names)
|
||||
return Response(207, text=f"<d:multistatus>{body}</d:multistatus>")
|
||||
if method == "PUT":
|
||||
self.puts.append(path)
|
||||
self.files[path] = content or b""
|
||||
return Response(201)
|
||||
if method == "GET":
|
||||
if path in self.files:
|
||||
return Response(200, content=self.files[path])
|
||||
return Response(404)
|
||||
if method == "MKCOL":
|
||||
return Response(201)
|
||||
return Response(200)
|
||||
def __exit__(self, *a):
|
||||
return False
|
||||
|
||||
def post(self, url, **kw):
|
||||
kw["headers"] = {**(kw.get("headers") or {}), "X-Test-IP": state["ip"]}
|
||||
return c.post(url, **kw)
|
||||
|
||||
def get(self, url, **kw):
|
||||
kw["headers"] = {**(kw.get("headers") or {}), "X-Test-IP": state["ip"]}
|
||||
return c.get(url, **kw)
|
||||
|
||||
monkeypatch.setattr(share, "_http", lambda: _Client())
|
||||
yield c, state
|
||||
with database.session() as db:
|
||||
_clean(db)
|
||||
share.reset_library_cache()
|
||||
local.reset()
|
||||
|
||||
|
||||
def _install(monkeypatch, dav):
|
||||
"""把 share.py 里的 httpx 模块函数换成 WebDAV 仿真"""
|
||||
import httpx
|
||||
def _as_new_instance(state, ip):
|
||||
"""换成另一台实例：清掉本机保存的注册信息，换来源地址"""
|
||||
with database.session() as db:
|
||||
for k in ("share_client_instance", "share_client_token", "share_client_hub"):
|
||||
row = db.get(Setting, k)
|
||||
if row is not None:
|
||||
db.delete(row)
|
||||
db.commit()
|
||||
share._client.update(votes_ts=0, etag="")
|
||||
state["ip"] = ip
|
||||
|
||||
from app import share as share_mod
|
||||
|
||||
monkeypatch.setattr(share_mod.httpx, "put", lambda url, content=None, **kw: dav.handle("PUT", url, content))
|
||||
monkeypatch.setattr(share_mod.httpx, "get", lambda url, **kw: dav.handle("GET", url))
|
||||
monkeypatch.setattr(share_mod.httpx, "request", lambda method, url, **kw: dav.handle(method, url))
|
||||
del httpx
|
||||
def _seed(db, rows):
|
||||
for dom, app, b in rows:
|
||||
db.add(HostDomainDay(ts=DAY, host=HOST, domain=dom, app=app, base=None, bytes=b))
|
||||
db.commit()
|
||||
|
||||
|
||||
def _client_cycle():
|
||||
with database.session() as db:
|
||||
res = share.client_cycle(engine, db)
|
||||
db.commit()
|
||||
return res
|
||||
|
||||
|
||||
def test_export_filters_private():
|
||||
import app.database as database
|
||||
database.connect_from_saved(force=True)
|
||||
s = database.SessionLocal()
|
||||
try:
|
||||
s.query(HostDomainDay).filter(HostDomainDay.host == HOST).delete(synchronize_session=False)
|
||||
s.add(HostDomainDay(ts=DAY, host=HOST, domain="93.184.216.34", app="douyin", base=None, bytes=800))
|
||||
s.add(HostDomainDay(ts=DAY, host=HOST, domain="192.168.8.5", app="douyin", base=None, bytes=999)) # 私网
|
||||
s.add(HostDomainDay(ts=DAY, host=HOST, domain="239.255.255.250", app="douyin", base=None, bytes=999)) # 组播
|
||||
s.commit()
|
||||
eng = _engine()
|
||||
from app import share as share_mod
|
||||
entries, _domains = share_mod.export_entries(eng, s)
|
||||
ips = {e["ip"] for e in entries}
|
||||
assert "93.184.216.34" in ips
|
||||
assert "192.168.8.5" not in ips and "239.255.255.250" not in ips
|
||||
finally:
|
||||
s.query(HostDomainDay).filter(HostDomainDay.host == HOST).delete(synchronize_session=False)
|
||||
s.commit()
|
||||
s.close()
|
||||
with database.session() as db:
|
||||
_clean(db)
|
||||
_seed(db, [("93.184.216.34", "douyin", 800), ("192.168.8.5", "douyin", 999),
|
||||
("239.255.255.250", "douyin", 999), ("v.douyin.com", "douyin", 50)])
|
||||
entries, domains = share.export_entries(engine, db)
|
||||
_clean(db)
|
||||
ips = {e["ip"] for e in entries}
|
||||
assert "93.184.216.34" in ips and not ips & {"192.168.8.5", "239.255.255.250"} # 私网 / 组播不出门
|
||||
assert "v.douyin.com" in {d["domain"] for d in domains}
|
||||
|
||||
|
||||
def test_webdav_aggregates_two_instances(monkeypatch):
|
||||
"""两个实例各写各的投票文件：≥2 个实例见过的映射进社区层；单实例见过的进不来"""
|
||||
import app.database as database
|
||||
database.connect_from_saved(force=True)
|
||||
dav = _FakeDav()
|
||||
_install(monkeypatch, dav)
|
||||
s = database.SessionLocal()
|
||||
try:
|
||||
s.query(HostDomainDay).filter(HostDomainDay.host == HOST).delete(synchronize_session=False)
|
||||
s.commit()
|
||||
eng = _engine()
|
||||
# A 的本机实测：1.2.3.4 → douyin（进入自己的投票文件）
|
||||
s.add(HostDomainDay(ts=DAY, host=HOST, domain="1.2.3.4", app="douyin", base=None, bytes=500))
|
||||
s.commit()
|
||||
# 第一轮：A 写入自己的投票文件；此时没有其它文件，社区层为空
|
||||
with database.session() as db:
|
||||
res1 = webdav_cycle(eng, db, BASE, "", "")
|
||||
db.commit()
|
||||
assert res1["deployed"] == 1 and res1["online"] == 1
|
||||
assert res1["community_ips"] == 0
|
||||
# B 的文件出现在 NAS 上：1.2.3.4 → douyin（与 A 相同）+ 5.6.7.8 → bilibili（只有 B 见过）
|
||||
dav.files["votes/IID_B.json"] = json.dumps({
|
||||
"version": "0.6.20", "ts": int(time.time()),
|
||||
"entries": [{"ip": "1.2.3.4", "app": "douyin", "bytes": 100},
|
||||
{"ip": "5.6.7.8", "app": "bilibili", "bytes": 200}],
|
||||
"domains": []}).encode()
|
||||
dav.files["deployments/IID_B.json"] = json.dumps(
|
||||
{"version": "0.6.20", "ts": int(time.time())}).encode()
|
||||
|
||||
# 第二轮：A 读 B 的文件并聚合
|
||||
with database.session() as db:
|
||||
res2 = webdav_cycle(eng, db, BASE, "", "")
|
||||
db.commit()
|
||||
assert res2["deployed"] == 2 and res2["online"] == 2
|
||||
assert res2["community_ips"] == 1 # 1.2.3.4 两家都见过
|
||||
assert local.ip_app("1.2.3.4") == "douyin"
|
||||
assert local.ip_app("5.6.7.8") is None # 单实例见过的进不了社区层
|
||||
assert len(dav.puts) == 4 # 2 轮 × (deployments + votes)
|
||||
finally:
|
||||
s.query(HostDomainDay).filter(HostDomainDay.host == HOST).delete(synchronize_session=False)
|
||||
s.commit()
|
||||
s.close()
|
||||
local.reset()
|
||||
dav.files.clear()
|
||||
dav.puts.clear()
|
||||
def test_client_ip_behind_proxy():
|
||||
# 直连公网：用 TCP 对端，伪造的 X-Forwarded-For 无效
|
||||
assert share.client_ip("8.8.8.8", "1.1.1.1", None) == "8.8.8.8"
|
||||
# 反向代理在内网：取代理追加在最右边的公网地址
|
||||
assert share.client_ip("192.168.8.1", "1.1.1.1, 9.9.9.9", None) == "9.9.9.9"
|
||||
assert share.client_ip("127.0.0.1", "10.0.0.5", "9.9.9.9") == "9.9.9.9"
|
||||
|
||||
|
||||
def test_webdav_single_instance_no_community(monkeypatch):
|
||||
def test_two_networks_publish_end_to_end(hub):
|
||||
c, state = hub
|
||||
with database.session() as db:
|
||||
_seed(db, [("1.2.3.4", "douyin", 500), ("v.douyin.com", "douyin", 300)])
|
||||
|
||||
# 第一台：注册 + 上报；只有一个网络见过，社区库为空
|
||||
res = _client_cycle()
|
||||
assert res["total"] == 1 and res["online"] == 1 and res["community_ips"] == 0
|
||||
assert share.sync_state()["ok"] is True and local.ip_app("1.2.3.4") is None
|
||||
|
||||
# 第二台（另一个网络）投了同样的结论：两网一致 → 进社区库，本机自学习的社区层生效
|
||||
_as_new_instance(state, "203.0.113.50")
|
||||
res = _client_cycle()
|
||||
assert res["total"] == 2 and res["community_ips"] >= 1 and res["community_domains"] >= 1
|
||||
assert local.ip_app("1.2.3.4") == "douyin"
|
||||
assert local.domain_app("v.douyin.com") == "douyin"
|
||||
|
||||
# 社区库没变化：带 ETag 再拉是 304，结果沿用；缓存文件已写，重启后可直接加载
|
||||
etag = share._client["etag"]
|
||||
assert etag and c.get(HUB + share.API + "/library", headers={"If-None-Match": etag}).status_code == 304
|
||||
local.reset()
|
||||
share.load_cached_library()
|
||||
assert local.ip_app("1.2.3.4") == "douyin"
|
||||
|
||||
# 公开的部署量
|
||||
assert c.get(HUB + share.API + "/stats").json() == {"total": 2, "online": 2, "offline": 0}
|
||||
|
||||
|
||||
def test_same_network_counts_once(hub):
|
||||
"""一个人在同一网络里伪造多个实例：只算一票，进不了社区库"""
|
||||
c, state = hub
|
||||
with database.session() as db:
|
||||
_seed(db, [("5.6.7.8", "bilibili", 100)])
|
||||
_client_cycle()
|
||||
_as_new_instance(state, "198.51.100.77") # 同一个 /24
|
||||
res = _client_cycle()
|
||||
assert res["total"] == 2 and res["community_ips"] == 0 and local.ip_app("5.6.7.8") is None
|
||||
|
||||
|
||||
def test_majority_and_whitelist(hub):
|
||||
c, _state = hub
|
||||
with database.session() as db:
|
||||
rows = []
|
||||
for i, ip in enumerate(("1.0.0.1", "2.0.0.1", "3.0.0.1")):
|
||||
iid, _t = share.register(db, share.net_of(db, ip), "t")
|
||||
db.flush()
|
||||
rows.append(db.get(ShareDeployment, iid))
|
||||
cls = engine.classifier
|
||||
share.report(db, rows[0], rows[0].net, {"entries": [{"ip": "8.8.8.8", "app": "douyin", "bytes": 1}]}, cls)
|
||||
share.report(db, rows[1], rows[1].net, {"entries": [{"ip": "8.8.8.8", "app": "douyin", "bytes": 1}]}, cls)
|
||||
share.report(db, rows[2], rows[2].net, {"entries": [{"ip": "8.8.8.8", "app": "bilibili", "bytes": 10 ** 9},
|
||||
{"ip": "9.9.9.9", "app": "made_up_app", "bytes": 1},
|
||||
{"ip": "10.0.0.1", "app": "douyin", "bytes": 1}]}, cls)
|
||||
db.commit()
|
||||
lib = share.build_library(db)
|
||||
n = db.query(ShareEntry).filter(ShareEntry.reporter == rows[2].instance).count()
|
||||
assert lib["entries"] == [{"ip": "8.8.8.8", "app": "douyin"}] # 两个网络对一个网络：按网络数，不按字节
|
||||
assert n == 1 # 编造的应用、私网地址被拒收
|
||||
|
||||
|
||||
def test_auth_and_limits(hub):
|
||||
c, _state = hub
|
||||
url = HUB + share.API
|
||||
assert c.post(url + "/report", json={"version": "x"}).status_code == 401
|
||||
reg = c.post(url + "/register", json={"version": "x"}, headers={"X-Test-IP": "192.0.2.1"}).json()
|
||||
bad = {"Authorization": f"Bearer {reg['instance']}:wrong"}
|
||||
assert c.post(url + "/report", json={"version": "x"}, headers=bad).status_code == 401
|
||||
ok = {"Authorization": f"Bearer {reg['instance']}:{reg['token']}", "Content-Encoding": "gzip"}
|
||||
assert c.post(url + "/report", content=gzip.compress(json.dumps({"version": "x"}).encode()), headers=ok).json()["ok"]
|
||||
# 解压后超限：拒收（防压缩炸弹）
|
||||
bomb = gzip.compress(b" " * (share.MAX_INFLATED + 10))
|
||||
assert c.post(url + "/report", content=bomb, headers=ok).status_code == 413
|
||||
# 同一网络一天注册超限
|
||||
for i in range(share.REGISTER_PER_NET):
|
||||
assert c.post(url + "/register", json={}, headers={"X-Test-IP": f"100.20.30.{i}"}).status_code == 200
|
||||
assert c.post(url + "/register", json={}, headers={"X-Test-IP": "100.20.30.200"}).status_code == 429
|
||||
|
||||
|
||||
def test_hub_counts_itself_in_process(hub):
|
||||
"""主站自己的投票在进程内计入（不经网络）：主站 + 一个普通实例两网一致即发布"""
|
||||
c, state = hub
|
||||
with database.session() as db:
|
||||
_seed(db, [("4.4.4.4", "douyin", 100)])
|
||||
res = share.hub_cycle(engine, db)
|
||||
db.commit()
|
||||
assert res["total"] == 1 and res["community_ips"] == 0
|
||||
assert local.ip_app("4.4.4.4") is None
|
||||
_client_cycle()
|
||||
with database.session() as db:
|
||||
share._lib["dirty"] = True
|
||||
res = share.hub_cycle(engine, db)
|
||||
db.commit()
|
||||
assert res["total"] == 2 and res["community_ips"] >= 1 and local.ip_app("4.4.4.4") == "douyin"
|
||||
|
||||
|
||||
def test_non_hub_404(monkeypatch):
|
||||
monkeypatch.setattr(get_settings(), "share_hub", False)
|
||||
c = TestClient(app)
|
||||
for path in ("/stats", "/library"):
|
||||
assert c.get(share.API + path).status_code == 404
|
||||
assert c.post(share.API + "/register", json={}).status_code == 404
|
||||
|
||||
|
||||
def test_client_error_is_reported(monkeypatch):
|
||||
"""主站不可达：设置页能看到原因，社区库沿用缓存"""
|
||||
import httpx
|
||||
|
||||
dav = _FakeDav()
|
||||
_install(monkeypatch, dav)
|
||||
eng = _engine()
|
||||
try:
|
||||
local.reset()
|
||||
import app.database as database
|
||||
database.connect_from_saved(force=True)
|
||||
s = database.SessionLocal()
|
||||
with database.session() as db:
|
||||
res = webdav_cycle(eng, db, BASE, "", "")
|
||||
db.commit()
|
||||
assert res["deployed"] == 1 and res["online"] == 1
|
||||
assert res["community_ips"] == 0 # 没有其它实例 → 社区层空（本地实测本来就优先）
|
||||
finally:
|
||||
s.close()
|
||||
local.reset()
|
||||
dav.files.clear()
|
||||
database.connect_from_saved(force=True)
|
||||
monkeypatch.setattr(get_settings(), "share_hub", False)
|
||||
monkeypatch.setattr(get_settings(), "share_hub_url", "http://127.0.0.1:9")
|
||||
|
||||
class _Down:
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, *a):
|
||||
return False
|
||||
|
||||
def post(self, url, **kw):
|
||||
raise httpx.ConnectError("connection refused")
|
||||
get = post
|
||||
|
||||
monkeypatch.setattr(share, "_http", lambda: _Down())
|
||||
with database.session() as db:
|
||||
with pytest.raises(httpx.ConnectError):
|
||||
share.cycle(engine, db)
|
||||
st = share.sync_state()
|
||||
assert st["ok"] is False and "ConnectError" in st["error"]
|
||||
|
||||
|
||||
def test_no_ipv6_egress_message():
|
||||
"""实例不需要公网 IP，但主站只有 IPv6：容器没有 IPv6 出口时给出能照着做的提示"""
|
||||
import httpx
|
||||
msg = share._err(httpx.ConnectError("[Errno 101] Network is unreachable"))
|
||||
assert "IPv6" in msg and "Docker" in msg
|
||||
@@ -92,10 +92,32 @@ def test_poll_success_populates_router_state(monkeypatch):
|
||||
assert chk["checked"] and sni["problems"] == ["port"] and sni["target_port"] == 37880
|
||||
# SNI 规则存在、只是端口错：建议改端口，不建议重复新增；但缺 UDP（QUIC）规则
|
||||
assert "rule_problem" in chk["issues"] and "no_sni" not in chk["issues"] and "no_quic" in chk["issues"]
|
||||
# 路由器有 IPv6 公网地址，嗅探规则却只有 IPv4 的一份
|
||||
assert "no_ipv6" in chk["issues"] and "fasttrack" not in chk["issues"]
|
||||
assert not codes["rules"]["ok"] and codes["rules"]["params"]["wrong"] == [37880]
|
||||
leases = c.get("/api/dhcp/leases", headers={"Authorization": f"Bearer {tok}"}).json()
|
||||
assert leases["leases"][0]["mac"] == "AA:BB:CC:DD:EE:31" and leases["leases"][0]["host_name"] == "iPhone"
|
||||
|
||||
def test_fasttrack_check_respects_connection_bytes(monkeypatch):
|
||||
"""FastTrack 开着：规则没限定 connection-bytes 要提示；已经限定（连接开头不走 FastTrack）就不再提示"""
|
||||
_use_mock(monkeypatch)
|
||||
monkeypatch.setitem(engine.stats, "dns_answers", 5)
|
||||
monkeypatch.setitem(ROS, "/rest/ip/settings", {"ipv4-fasttrack-active": "true"})
|
||||
rule = {".id": "*2", "action": "fasttrack-connection", "chain": "forward",
|
||||
"connection-state": "established,related", "disabled": "false"}
|
||||
monkeypatch.setitem(ROS, "/rest/ip/firewall/filter", [rule])
|
||||
poller.poll()
|
||||
assert "fasttrack" in poller.check()["issues"]
|
||||
monkeypatch.setitem(ROS, "/rest/ip/firewall/filter", [{**rule, "connection-bytes": "10000-0"}])
|
||||
poller.poll()
|
||||
chk = poller.check()
|
||||
assert "fasttrack" not in chk["issues"] and chk["fasttrack_limited"]
|
||||
# 读不到 filter 表（只读账号没权限）：按旧办法提示
|
||||
monkeypatch.delitem(ROS, "/rest/ip/firewall/filter")
|
||||
poller.poll()
|
||||
assert "fasttrack" in poller.check()["issues"]
|
||||
|
||||
|
||||
def test_poll_401_gives_actionable_error_and_keeps_status_usable(monkeypatch):
|
||||
_use_mock(monkeypatch, status=401)
|
||||
poller.poll()
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
"""所有无路径参数的 GET 接口都不能返回 500（0.6.20 的部署量接口调用了已删除的函数，上线后才发现）"""
|
||||
from fastapi.routing import APIRoute
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from app.main import app
|
||||
|
||||
SKIP = {"/api/system/update"} # 会访问 GitHub
|
||||
|
||||
|
||||
def _h(c):
|
||||
if not c.get("/api/auth/status").json()["initialized"]:
|
||||
c.post("/api/auth/setup", json={"username": "admin", "password": "secret1"})
|
||||
return {"Authorization": "Bearer " + c.post("/api/auth/login", json={"username": "admin", "password": "secret1"}).json()["token"]}
|
||||
|
||||
|
||||
def test_get_routes_no_500():
|
||||
with TestClient(app) as c:
|
||||
h = _h(c)
|
||||
bad = []
|
||||
for r in app.routes:
|
||||
if not isinstance(r, APIRoute) or "GET" not in r.methods or "{" in r.path:
|
||||
continue
|
||||
if not r.path.startswith("/api/") or r.path in SKIP:
|
||||
continue
|
||||
resp = c.get(r.path, headers=h)
|
||||
if resp.status_code >= 500:
|
||||
bad.append((r.path, resp.status_code, resp.text[:120]))
|
||||
assert not bad, bad
|
||||
@@ -53,3 +53,10 @@ services:
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 20
|
||||
|
||||
# 社区共享库：社区主站目前只有 IPv6 地址，而 Docker 默认网桥没有 IPv6 出口，容器会连不上主站
|
||||
# （设置页显示“连接失败：本机没有 IPv6 出口”）。Docker 27 及以上取消下面 3 行注释即可（自动分配地址并做 NAT）；
|
||||
# 更老的 Docker（群晖 / 部分 NAS 自带的版本）请改用 Linux 一键安装，或在 daemon.json 里开启 ipv6 + ip6tables。
|
||||
# networks:
|
||||
# default:
|
||||
# enable_ipv6: true
|
||||
@@ -26,3 +26,10 @@ services:
|
||||
# 旁路代理（sing-box）和内网 DNS（MosDNS）在网页 系统设置 → 外部数据源 里填写，不需要写在这里
|
||||
volumes:
|
||||
- ./data:/app/data # 保存数据库连接配置，务必保留这个卷
|
||||
|
||||
# 社区共享库：社区主站目前只有 IPv6 地址，而 Docker 默认网桥没有 IPv6 出口，容器会连不上主站
|
||||
# （设置页显示“连接失败：本机没有 IPv6 出口”）。Docker 27 及以上取消下面 3 行注释即可（自动分配地址并做 NAT）；
|
||||
# 更老的 Docker（群晖 / 部分 NAS 自带的版本）请改用 Linux 一键安装，或在 daemon.json 里开启 ipv6 + ip6tables。
|
||||
# networks:
|
||||
# default:
|
||||
# enable_ipv6: true
|
||||
@@ -155,13 +155,13 @@ export default {
|
||||
agUrl: 'AdGuard Home address', agUser: 'Login username (leave empty if none)', agPassword: 'Login password',
|
||||
agKept: 'Set; leave empty to keep', agNone: 'Leave empty if none', agClear: 'Clear password',
|
||||
agHint: 'AdGuard Home query log must be on (enabled by default); empty username and password mean unauthenticated access.',
|
||||
shareTitle: 'Community library (WebDAV)',
|
||||
deploys: 'Global deployments', relaysLabel: 'Available relays', deployed: '{n} deployed', online: '{n} online', offline: '{n} offline', deployHint: 'from the relay, refreshed hourly',
|
||||
shareEnabled: 'Share the learned library (on by default)', shareNoUrl: 'No relay address set', shareWait: 'Configured, waiting for the first sync',
|
||||
shareTitle: 'Community library',
|
||||
deploys: 'Global deployments', deployed: '{n} deployed', online: '{n} online', offline: '{n} offline', deployHint: 'from the community hub, refreshed every 30 minutes',
|
||||
shareEnabled: 'Join community sharing (on by default)', shareWait: 'Waiting for the first sync', shareHub: 'This instance is the community hub',
|
||||
shareNets: 'Source networks', shareNetsWarn: 'Several instances but only 1 source network: the reverse proxy in front of the hub does not pass X-Forwarded-For, so every instance looks like the same network and nothing reaches the vote threshold. Enable forwarding the client IP in the proxy.',
|
||||
shareHubUrl: 'Community hub', shareHubHint: 'built in, nothing to fill in; set SHARE_HUB_URL to run a private community',
|
||||
communitySize: 'Community library', communityDetail: '{ip} IP memories · {dom} domain memories',
|
||||
shareWhy: 'Every deployment learns different IP→app mappings. Pooling anonymous votes makes the shared identification sharper for everyone. Reports contain remote-IP→app votes only — no local devices, no domain records, no custom apps; private ranges are stripped.',
|
||||
shareUrl: 'WebDAV folder URL', shareUser: 'WebDAV username', sharePassword: 'WebDAV password',
|
||||
shareHint: 'Enable WebDAV on your NAS and create a folder (e.g. ros-dpi), then put its URL here. Every 30 minutes: your anonymous votes are written, votes of other instances are read and aggregated locally (entries seen by 2+ instances win). HTTPS or LAN-only recommended.',
|
||||
shareWhy: 'Every deployment learns different IP/domain→app mappings. Anonymous votes are pooled on the community hub; a mapping is published only when instances on at least 2 different networks agree. Reports contain no local devices, no access records, no custom apps; private ranges are stripped. Turning this off stops reporting and stops using the community library.',
|
||||
asn: 'IP ownership (ASN)', asnOn: 'On',
|
||||
asnWhy: 'Domain-less traffic is attributed to the owning company by IP (Tencent, Alibaba, Apple, Netflix…). Data from iptoasn.com (public domain), refreshed daily; off without internet access.',
|
||||
asnRows: '{n} ranges, updated {d}.', connected: 'Connected', failed: 'Failed', off: 'Off',
|
||||
@@ -177,7 +177,8 @@ export default {
|
||||
issue: {
|
||||
no_dns: 'No working DNS sniffing rule.', no_sni: 'No working SNI rule: HTTPS / QUIC handshakes never reach ROS-DPI.',
|
||||
no_quic: 'Only a TCP SNI rule; UDP 443 (QUIC / HTTP3) is missing, so QUIC-heavy apps such as Douyin and YouTube are not identified.',
|
||||
fasttrack: 'FastTrack is on: fast-tracked connections skip mangle, so handshakes are not mirrored. Add connection-bytes=10000-0 to the FastTrack rule.',
|
||||
fasttrack: 'FastTrack is on and its rule has no connection-bytes limit: fast-tracked connections skip mangle, so handshakes are not mirrored and much HTTPS falls back to IP-only buckets. The command below keeps the first 10 KB of each connection off FastTrack; CPU cost barely changes.',
|
||||
no_ipv6: 'The router has public IPv6 addresses but sniff rules exist for IPv4 only: DNS answers and HTTPS / QUIC handshakes over IPv6 are invisible. The commands below add the same rules under /ipv6 firewall mangle; if sniff-target is rejected, use the IPv6 address of ROS-DPI.',
|
||||
},
|
||||
},
|
||||
audit: {
|
||||
|
||||
@@ -155,13 +155,13 @@ export default {
|
||||
agUrl: 'AdGuard Home 地址', agUser: '登录用户名（没有可留空）', agPassword: '登录密码',
|
||||
agKept: '已设置，不修改请留空', agNone: '没有设置可留空', agClear: '清除密码',
|
||||
agHint: '需要 AdGuard Home 开启查询日志（默认开启）；账号和密码都留空表示免认证访问。',
|
||||
shareTitle: '社区共享库（WebDAV）',
|
||||
deploys: '全球部署量', relaysLabel: '可用中转', deployed: '已部署 {n}', online: '在线 {n}', offline: '离线 {n}', deployHint: '来自社区中转，每小时刷新',
|
||||
shareEnabled: '共享识别库（默认打开）', shareNoUrl: '未配置中转地址', shareWait: '已配置，等待首次同步',
|
||||
shareTitle: '社区共享库',
|
||||
deploys: '全球部署量', deployed: '已部署 {n}', online: '在线 {n}', offline: '离线 {n}', deployHint: '来自社区主站，每 30 分钟刷新',
|
||||
shareEnabled: '参与社区共享（默认打开）', shareWait: '等待首次同步', shareHub: '本实例是社区主站',
|
||||
shareNets: '来源网络', shareNetsWarn: '多个实例却只看到 1 个来源网络：主站前面的反向代理没有传 X-Forwarded-For，所有实例被当成同一网络，社区库凑不够票。请在反向代理里开启传递客户端 IP。',
|
||||
shareHubUrl: '社区主站', shareHubHint: '项目内置，不需要填写；自建私有社区时用环境变量 SHARE_HUB_URL 修改',
|
||||
communitySize: '社区库现有', communityDetail: 'IP 记忆 {ip} 条 · 域名记忆 {dom} 条',
|
||||
shareWhy: '每个部署实例的自学习库都不一样：把各自实测的「远端 IP → 应用」匿名投票汇总到一起，大家共同的识别就更准。上报内容不含本机终端、域名记录与自定义应用，私网地址自动剔除。',
|
||||
shareUrl: 'WebDAV 共享文件夹地址', shareUser: 'WebDAV 账号', sharePassword: 'WebDAV 密码',
|
||||
shareHint: '在 NAS 上开启 WebDAV 并建一个共享文件夹（如 ros-dpi），把地址填到这里；每 30 分钟同步一次：写入自己的匿名投票、读取其它实例的投票并在本地聚合（至少 2 台见过才采用）。飞牛 / 群晖在应用中心开启 WebDAV 即可；建议 HTTPS 或仅内网使用。',
|
||||
shareWhy: '每个部署实例的自学习库都不一样：把各自实测的「远端 IP / 域名 → 应用」匿名投票汇总到社区主站，至少 2 个不同网络的实例都这样认的才进入社区库，大家的识别就更准。上报内容不含本机终端、访问记录与自定义应用，私网地址自动剔除；关闭后不再上报，也不再使用社区库。',
|
||||
asn: 'IP 归属库（ASN）', asnOn: '已启用',
|
||||
asnWhy: '没有域名的流量按 IP 归属认出公司（腾讯、阿里、苹果、Netflix……）。数据来自 iptoasn.com（公有领域），容器每天自动更新；没有外网时不启用。',
|
||||
asnRows: '共 {n} 条，更新于 {d}。', connected: '已连接', failed: '连接失败', off: '未启用',
|
||||
@@ -177,7 +177,8 @@ export default {
|
||||
issue: {
|
||||
no_dns: '没有生效的 DNS 嗅探规则。', no_sni: '没有生效的 SNI 规则：HTTPS / QUIC 握手到不了 ROS-DPI。',
|
||||
no_quic: '只有 TCP 的 SNI 规则，缺少 UDP 443（QUIC / HTTP3），抖音、YouTube 等大量走 QUIC 的流量认不出来。',
|
||||
fasttrack: 'FastTrack 已启用：被 FastTrack 的连接不经过 mangle，握手包镜像不到。可以给 FastTrack 规则加 connection-bytes=10000-0。',
|
||||
fasttrack: 'FastTrack 已启用，且规则没有限定 connection-bytes：被 FastTrack 的连接不经过 mangle，握手包镜像不到，大量 HTTPS 只能按 IP 归到「国内网络」。下方命令让每条连接的前 10KB 不走 FastTrack，CPU 占用几乎不变。',
|
||||
no_ipv6: '路由器有 IPv6 公网地址，但嗅探规则只有 IPv4 的：走 IPv6 的 DNS 应答和 HTTPS / QUIC 握手都看不到（国内运营商默认下发 IPv6，这部分流量通常很大）。下方命令在 /ipv6 firewall mangle 补一份；若提示 sniff-target 无效，改填 ROS-DPI 的 IPv6 地址。',
|
||||
},
|
||||
},
|
||||
audit: {
|
||||
|
||||
@@ -263,7 +263,8 @@ async function reclassify() {
|
||||
reclassing.value = true
|
||||
reclassRes.value = ''
|
||||
try {
|
||||
const r = await http.post('/rules/reclassify')
|
||||
// 数据多时要一两分钟（审计报告 P1-3：实测 80 秒），默认 20 秒超时会误报失败而后端其实还在跑
|
||||
const r = await http.post('/rules/reclassify', null, { timeout: 600000 })
|
||||
reclassRes.value = t('rules.reclassifyDone', { b: bytes(r.bytes_moved), s: r.seconds })
|
||||
} catch (e) { reclassRes.value = e.message } finally { reclassing.value = false }
|
||||
}
|
||||
|
||||
@@ -163,38 +163,34 @@
|
||||
</div>
|
||||
<div class="src-box">
|
||||
<div class="sec">{{ t('sources.shareTitle') }}<span class="sp" />
|
||||
<span v-if="!s.share_enabled" class="pill grey">{{ t('sources.off') }}</span>
|
||||
<template v-else-if="!src.share_url"><span class="pill grey">{{ t('sources.shareNoUrl') }}</span></template>
|
||||
<template v-else>
|
||||
<span v-if="deploys && deploys.sync && deploys.sync.ts" class="pill" :class="deploys.sync.ok ? 'ok' : 'bad'">
|
||||
{{ deploys.sync.ok ? t('sources.connected') : t('sources.failed') }}</span>
|
||||
<span v-else class="pill grey">{{ t('sources.shareWait') }}</span>
|
||||
</template></div>
|
||||
<span v-if="deploys && deploys.role === 'hub'" class="pill ok">{{ t('sources.shareHub') }}</span>
|
||||
<span v-else-if="!s.share_enabled" class="pill grey">{{ t('sources.off') }}</span>
|
||||
<span v-else-if="deploys && deploys.sync.ts" class="pill" :class="deploys.sync.ok ? 'ok' : 'bad'">
|
||||
{{ deploys.sync.ok ? t('sources.connected') : t('sources.failed') }}</span>
|
||||
<span v-else class="pill grey">{{ t('sources.shareWait') }}</span></div>
|
||||
<p class="faint small">{{ t('sources.shareWhy') }}</p>
|
||||
<label class="chk" style="display:flex;gap:6px;align-items:center;cursor:pointer">
|
||||
<input type="checkbox" :checked="s.share_enabled" @change="src.share_enabled = $event.target.checked" /> {{ t('sources.shareEnabled') }}
|
||||
<input type="checkbox" :checked="src.share_enabled" @change="src.share_enabled = $event.target.checked" /> {{ t('sources.shareEnabled') }}
|
||||
</label>
|
||||
<label for="share-url">{{ t('sources.shareUrl') }}</label>
|
||||
<input id="share-url" class="input addr" v-model.trim="src.share_url" placeholder="http://192.168.8.10:5005/ros-dpi" autocomplete="off" />
|
||||
<label for="share-user">{{ t('sources.shareUser') }}</label>
|
||||
<input id="share-user" class="input" v-model.trim="src.share_user" autocomplete="off" />
|
||||
<label for="share-password">{{ t('sources.sharePassword') }}</label>
|
||||
<div class="row">
|
||||
<input id="share-password" class="input" type="password" v-model="src.share_password" autocomplete="new-password"
|
||||
:placeholder="s.has_webdav_password ? t('sources.agKept') : t('sources.agNone')" />
|
||||
<button v-if="s.has_webdav_password" type="button" class="btn sm" @click="clearSharePassword">{{ t('sources.agClear') }}</button>
|
||||
<div class="small" style="margin-top:6px" v-if="deploys">
|
||||
<b>{{ t('sources.shareHubUrl') }}</b>：<span class="addr">{{ deploys.hub_url || '—' }}</span>
|
||||
<span class="faint small" v-if="deploys.role !== 'hub'">（{{ t('sources.shareHubHint') }}）</span>
|
||||
</div>
|
||||
<div class="faint small">{{ t('sources.shareHint') }}</div>
|
||||
<div class="small" style="margin-top:8px" v-if="deploys && deploys.sync && deploys.sync.ts">
|
||||
<div class="err small" v-if="deploys && deploys.sync.ts && !deploys.sync.ok && deploys.sync.error">{{ deploys.sync.error }}</div>
|
||||
<div class="small" style="margin-top:8px" v-if="deploys && deploys.sync.ts">
|
||||
<b>{{ t('sources.communitySize') }}</b>：{{ t('sources.communityDetail', { ip: (deploys.sync.community_ips || 0).toLocaleString(), dom: (deploys.sync.community_domains || 0).toLocaleString() }) }}
|
||||
<span class="faint small">（{{ t('settings.last') }} {{ ago(deploys.sync.ts) }}）</span>
|
||||
</div>
|
||||
<div class="small" style="margin-top:8px" v-if="deploys">
|
||||
<b>{{ t('sources.deploys') }}</b>：{{ t('sources.deployed', { n: deploys.total }) }} ·
|
||||
<span style="color:var(--up)">{{ t('sources.online', { n: deploys.online }) }}</span> ·
|
||||
{{ t('sources.offline', { n: deploys.offline }) }}
|
||||
<div class="small" style="margin-top:8px" v-if="deploys && deploys.sync.total != null">
|
||||
<b>{{ t('sources.deploys') }}</b>：{{ t('sources.deployed', { n: deploys.sync.total }) }} ·
|
||||
<span style="color:var(--up)">{{ t('sources.online', { n: deploys.sync.online }) }}</span> ·
|
||||
{{ t('sources.offline', { n: deploys.sync.offline }) }}
|
||||
<span class="faint small">（{{ t('sources.deployHint') }}）</span>
|
||||
</div>
|
||||
<div class="small" style="margin-top:6px" v-if="deploys && deploys.role === 'hub'">
|
||||
<b>{{ t('sources.shareNets') }}</b>：{{ deploys.sync.nets }}
|
||||
<div class="err small" v-if="deploys.sync.total > 2 && deploys.sync.nets <= 1">{{ t('sources.shareNetsWarn') }}</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="acts">
|
||||
<span class="err" v-if="srcErr">{{ srcErr }}</span><span class="sp" />
|
||||
@@ -330,7 +326,7 @@ onMounted(async () => {
|
||||
})
|
||||
|
||||
// ---------- 外部数据源 ----------
|
||||
const src = reactive({ singbox_url: '', singbox_secret: '', proxy_ips: '', mosdns_url: '', adguard_url: '', adguard_username: '', adguard_password: '', share_url: '', share_enabled: true, share_user: '', share_password: '' })
|
||||
const src = reactive({ singbox_url: '', singbox_secret: '', proxy_ips: '', mosdns_url: '', adguard_url: '', adguard_username: '', adguard_password: '', share_enabled: true })
|
||||
const deploys = ref(null)
|
||||
const srcErr = ref('')
|
||||
const srcSaved = ref(false)
|
||||
@@ -339,7 +335,7 @@ function fillSources() {
|
||||
const i = s.value.integrations
|
||||
Object.assign(src, { singbox_url: i.singbox_url, singbox_secret: '', proxy_ips: i.proxy_ips, mosdns_url: i.mosdns_url,
|
||||
adguard_url: i.adguard_url, adguard_username: i.adguard_username, adguard_password: '',
|
||||
share_url: i.webdav_url, share_user: i.webdav_user || '', share_enabled: !!s.value.share_enabled, share_password: '' })
|
||||
share_enabled: !!s.value.share_enabled })
|
||||
}
|
||||
async function saveSources() {
|
||||
srcErr.value = ''
|
||||
@@ -347,23 +343,20 @@ async function saveSources() {
|
||||
// 密钥留空表示不修改；要清除用“清除密钥/密码”按钮
|
||||
const body = { singbox_url: src.singbox_url, proxy_ips: src.proxy_ips, mosdns_url: src.mosdns_url,
|
||||
adguard_url: src.adguard_url, adguard_username: src.adguard_username,
|
||||
share_url: src.share_url, share_user: src.share_user }
|
||||
share_enabled: src.share_enabled }
|
||||
if (src.singbox_secret) body.singbox_secret = src.singbox_secret
|
||||
if (src.adguard_password) body.adguard_password = src.adguard_password
|
||||
if (src.share_password) body.webdav_password = src.share_password
|
||||
s.value = await http.put('/settings', body)
|
||||
fillSources()
|
||||
srcSaved.value = true
|
||||
setTimeout(() => (srcSaved.value = false), 1500)
|
||||
await testSources()
|
||||
setTimeout(loadDeploys, 3000) // 打开共享开关后后台立刻同步一轮，稍后刷新状态
|
||||
} catch (e) { srcErr.value = e.message }
|
||||
}
|
||||
async function clearSecret() {
|
||||
try { s.value = await http.put('/settings', { singbox_secret: '' }); fillSources() } catch (e) { srcErr.value = e.message }
|
||||
}
|
||||
async function clearSharePassword() {
|
||||
try { s.value = await http.put('/settings', { webdav_password: '' }); fillSources() } catch (e) { srcErr.value = e.message }
|
||||
}
|
||||
async function clearAdguardPassword() {
|
||||
try { s.value = await http.put('/settings', { adguard_password: '' }); fillSources() } catch (e) { srcErr.value = e.message }
|
||||
}
|
||||
@@ -469,6 +462,18 @@ const fixes = computed(() => {
|
||||
} else if (c.issues.includes('no_quic')) {
|
||||
out.push(`/ip firewall mangle add chain=forward protocol=udp dst-port=443 connection-bytes=0-4000 ${tail}`)
|
||||
}
|
||||
if (c.issues.includes('no_ipv6')) {
|
||||
// IPv6 再加一份同样的 DNS / SNI 嗅探（镜像包仍发到 ROS-DPI 的 IPv4 地址）
|
||||
const dns = `action=sniff-tzsp sniff-target=${tgt} sniff-target-port=${c.port} comment=ROSDPI-DNS`
|
||||
out.push(`/ipv6 firewall mangle add chain=output protocol=udp src-port=53 ${dns}`)
|
||||
out.push(`/ipv6 firewall mangle add chain=forward protocol=udp src-port=53 ${dns}`)
|
||||
out.push(`/ipv6 firewall mangle add chain=forward protocol=tcp dst-port=443 connection-bytes=0-4000 ${tail}`)
|
||||
out.push(`/ipv6 firewall mangle add chain=forward protocol=udp dst-port=443 connection-bytes=0-4000 ${tail}`)
|
||||
}
|
||||
if (c.issues.includes('fasttrack')) {
|
||||
// 每条连接的前 10KB 走正常路径（握手能被镜像），之后照常 FastTrack
|
||||
out.push('/ip firewall filter set [find action=fasttrack-connection] connection-bytes=10000-0')
|
||||
}
|
||||
return [...new Set(out)]
|
||||
})
|
||||
const fixCopied = ref(false)
|
||||
|
||||
Reference in new issue
Block a user