diff --git a/.env.example b/.env.example index ad4db45..690a87c 100644 --- a/.env.example +++ b/.env.example @@ -75,8 +75,9 @@ LAN_CIDRS=192.168.88.0/24 # RETENTION_EVENT_DAYS=180 # 审计事件、操作日志 # 以上留存天数也可以在网页 系统设置 → 日志与留存 里改，网页优先 -# ---------- 社区共享库（debug16） ---------- -# 把本网络实测的「远端 IP → 应用」匿名投票上报给社区中转，并拉取社区库；只含远端 IP 和应用 ID， -# 不含本机终端、域名记录、自定义应用，私网地址剔除。总开关在网页 系统设置 → 外部数据源（默认打开）。 -# SHARE_URL=https://<社区中转地址> # 留空不参与；中转可以是任何一个开启了「共享中转」的实例 -# SHARE_TOKEN= # 中转设置了 SHARE_TOKEN 时才需要 +# ---------- 社区共享库 ---------- +# 把本网络实测的「远端 IP / 域名 → 应用」匿名投票汇总到项目自带的社区主站，并拉取社区库；不含本机终端、 +# 访问记录、自定义应用，私网地址剔除。开关在网页 系统设置 → 外部数据源（默认打开）。 +# SHARE_HUB_URL=https://rosdpi.suyijun.top:8443 # 社区主站（内置，一般不用改）；自建私有社区时改成自己的主站 +# SHARE_HUB=false # 只有主站自己设为 true +# SHARE_VERIFY_TLS=true # 私有主站用自签证书时才设 false diff --git a/CHANGELOG.md b/CHANGELOG.md index aa21782..fb30ad6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,30 @@ # Changelog +## [0.6.21] - 2026-10-25 + +社区共享库重做：项目自带社区主站，不再依赖 WebDAV / 第三方存储（debug13~22 复查） + +- **修复 0.6.20 的共享整体不可用**： + - 设置页打开就报 500：部署量接口调用了已删除的函数； + - 共享任务一次都没跑过：调用了已删除的 `sharing_on`，启动时报错被吞掉； + - 共享开关保存不上：前端没把开关发给后端。 +- **社区主站**（debug22 改为项目内实现）：主站和普通实例是同一套程序，主站部署设 `SHARE_HUB=true`，地址内置为 `https://rosdpi.suyijun.top:8443`（`SHARE_HUB_URL` 可改成自建的私有社区）。 + - 普通实例每 30 分钟心跳、每 6 小时附带一次匿名投票，拉社区库时没变化主站回 304；社区库缓存在本地，主站暂时不可达时继续可用。 + - 主站自己的投票在进程内计入，不再通过公网地址访问自己（旧中转在 NAT 回环不通时永远同步不上）。 + - 设置页只保留“参与社区共享”开关，显示主站地址、连接状态和失败原因、社区库规模、全球已部署 / 在线 / 离线；主站还显示来源网络数，反向代理没传客户端 IP 时提示。 +- **防冒充、防刷票**： + - 每个实例注册时领取令牌，别人无法冒充它或覆盖它的投票； + - 按**来源网络**计票（IPv4 /24、IPv6 /56，只存加盐哈希、不存来源 IP），至少 2 个不同网络的实例投给同一应用、并且票数领先才发布——同一网络伪造再多实例也只算一票； + - 同一网络每天限注册 5 个实例，投票每小时最多收一次，请求体和解压后大小都有上限（防压缩炸弹）； + - 应用白名单、域名格式、私网地址剔除照旧；主站证书恢复校验（`SHARE_VERIFY_TLS`，私有社区用自签证书时才关）。 +- **不需要公网 IP**：心跳、上报、拉库都是实例主动连主站。主站目前只有 IPv6 地址：Linux 一键安装直接可用；Docker 默认网桥没有 IPv6 出口，`docker-compose.yml` 末尾给了开启方法（Docker 27+），连不上时设置页直接提示原因。 +- 删除 WebDAV 共享和 0.6.15~0.6.19 的中转代码。旧中转留下的无令牌心跳不计入部署量，各实例升级后会重新注册。 +- **接入自检**： + - 路由器有 IPv6 公网地址、嗅探规则却只有 IPv4 的一份时提示，并给出 `/ipv6 firewall mangle` 的 4 条命令； + - 读取 `/ip/firewall/filter`，FastTrack 规则已经限定 connection-bytes 时不再误报，没限定时直接给出修改命令。主站点实测：FastTrack 开着、SNI 只解析到几十个握手，“国内网络”占 21%，主要原因就在这里。 +- 重算历史分类的请求超时从 20 秒放宽到 10 分钟（审计报告 P1-3：实测 80 秒，前端会误报失败）。 +- 新增接口烟雾测试（所有无路径参数的 GET 接口都不能返回 500），以及主站 / 客户端端到端测试。 + ## [0.6.20] - 2026-10-25 社区共享库改用 WebDAV（NAS 自带），取消中转模式（debug22） diff --git a/DOCKERHUB.md b/DOCKERHUB.md index 916532c..488916a 100644 --- a/DOCKERHUB.md +++ b/DOCKERHUB.md @@ -36,6 +36,7 @@ services: ## What's new +- **0.6.21**: 社区共享库重做——项目自带社区主站，不再需要 WebDAV / 第三方存储：令牌注册、按来源网络计票（≥2 个不同网络一致才发布，防伪造实例刷票）、ETag 增量拉取、本地缓存；修复 0.6.20 部署量接口 500、共享任务从未运行、开关保存不上。接入自检新增 IPv6 嗅探规则检查，FastTrack 已限定 connection-bytes 时不再误报，并直接给出修改命令。Community library rebuilt around a built-in hub (no WebDAV/third-party storage): per-instance tokens, votes counted per source network, ETag pulls, local cache; fixes the 0.6.20 deployments 500 and the never-running share job. Self-check now flags missing IPv6 sniff rules. - **0.6.12 / 0.6.11 / 0.6.10 / 0.6.9 / 0.6.8**: 自学习与共享系列。**本地自学习库**：本网络实测的「IP → 应用」（IPv6 按 /64 前缀聚合）与「域名 → 应用」长期记忆，每小时重建，DNS 缓存过期、DoH、裸 IP 流量都能按记忆归类；**社区共享库**：各部署实例把实测结论匿名共享（中转聚合、至少 2 个实例见过才发布），本地实测优先、社区只补空白；**全球部署量**：实例心跳上报，中转公开已部署 / 在线 / 离线数量；共享总开关默认打开；**一键自更新**：Linux 二进制部署在网页上检测并一键升级（自动下载、替换、重启服务）；**Linux 单文件二进制部署**：`install.sh` 一键安装 systemd 服务（SQLite 零配置起步）；修复落库卡死（代理长域名撑爆 access_log.remote 列导致每分钟落库失败、在线终端归零）；组播 / 广播流量不再计入统计；17 个 DPI 大类配色互异。 - **0.6.7**: fixes for the 0.6.5 / 0.6.6 history reclassification and rule auto-registration. Reclassification now only touches days before today and only upgrades vague rows (HTTPS / Chinese websites / unknown / CDN) to a specific app — it no longer downgrades apps identified live by IP range, port or ASN. New upstream blackmatrix7 directories are downloaded first and then assigned by a vote of the existing libraries. The manual dark theme now uses the 17 distinct group colors, and the AdGuard Home password is masked in the operation log. - **0.6.6**: AdGuard Home's newer query-log format (`{"data": [...]}`) is now read (0.6.5 always got 0 answers); new upstream rule directories / lists are registered automatically; history reclassification looks up the DNS cache for IP-only rows and runs automatically after library updates. @@ -119,7 +120,9 @@ Open `http://:8080`. The setup wizard has two steps. 打开网页后按两 | `ADMIN_PASSWORD` | empty | initial password for `admin` | | `ROS_URL` / `ROS_USER` / `ROS_PASSWORD` | empty | RouterOS REST + read-only user | | `SINGBOX_URL` / `MOSDNS_URL` / `ADGUARD_URL` | empty | external sources (Clash API, MosDNS audit, AdGuard Home query log) | -| `SHARE_URL` / `SHARE_TOKEN` | empty | community library relay (anonymous IP→app / domain→app votes) | +| `SHARE_HUB_URL` | `https://rosdpi.suyijun.top:8443` | community hub (built in); point it at your own hub for a private community | +| `SHARE_HUB` | `false` | run this instance as the community hub (only the hub sets `true`) | +| `SHARE_VERIFY_TLS` | `true` | verify the hub certificate; set `false` only for a private hub with a self-signed cert | | `TZ` | `Asia/Shanghai` | statistics time zone (weeks start Monday) | Full list: https://github.com/suyijun8182/ros-dpi#环境变量 diff --git a/README.md b/README.md index cf91487..b4c8c20 100644 --- a/README.md +++ b/README.md @@ -42,7 +42,7 @@ RouterOS ◀─REST 只读（可选）── 主机名、IP↔MAC、WAN 口速 - **域名 → 应用规则库，四层叠加**： 1. 公开库：内置手工规则库 + [v2fly](https://github.com/v2fly/domain-list-community)（约 2500 应用）+ [blackmatrix7](https://github.com/blackmatrix7/ios_rule_script)（自动登记上游新目录）+ Hagezi / UT1 分类列表 + 国内域名列表 / 官方 IP 段 / 国内 IP 段兜底； 2. **本地自学习库**：本网络实测的「IP → 应用」与「域名 → 应用」长期记忆——DNS 缓存过期、DoH、裸 IP 流量都能按记忆归类，每小时自动重建； - 3. **社区共享库**：各部署实例把实测结论匿名共享（至少 2 个实例见过才发布），本地没有结论时兜底； + 3. **社区共享库**：各部署实例把实测结论匿名汇总到项目自带的社区主站（至少 2 个不同网络的实例一致才发布），本地没有结论时兜底； 4. 以上都没有时，才归到 HTTPS / 国内网络这类笼统分类。 - 终端用 MAC 区分，IP 变化不影响统计。 @@ -186,7 +186,7 @@ EOF 1. **公开库**（自动更新）：内置手工库 + v2fly（每天自检更新）+ blackmatrix7（每周，上游新目录自动登记：先下载、再由现有识别库投票定归属）+ Hagezi / UT1 分类列表 + ASN（iptoasn）+ 国内域名列表 / 官方 IP 段 / 国内 IP 段兜底。 2. **本地自学习库**：本网络实测的「IP → 应用」（IPv6 按 /64 前缀聚合）与「域名 → 应用」长期记忆，每小时从数据库重建；DNS 缓存过期、DoH、裸 IP 流量都能按记忆归类。每次历史重算自动纠正它。 -3. **社区共享库**：各部署实例把实测结论匿名共享——只含远端 IP / 域名、应用 ID 和字节票数，不含本机终端、域名记录与自定义应用，私网地址剔除；中转聚合**至少 2 个实例见过**的映射才发布。系统设置 → 外部数据源 填中转地址即参与（总开关默认打开），也可把自己的实例开成中转（公网可达）。中转的 `/api/share/deployments` 公开显示全球已部署 / 在线 / 离线数量。 +3. **社区共享库**：项目自带社区主站，不依赖任何第三方存储。各实例每 30 分钟向主站心跳、每 6 小时上报一次匿名投票——只含远端 IP / 域名、应用 ID 和字节票数，不含本机终端、访问记录与自定义应用，私网地址剔除；主站按**来源网络**计票（IPv4 /24、IPv6 /56，只存加盐哈希），**至少 2 个不同网络的实例投给同一应用且票数领先**才发布，一个人伪造多个实例也只算一票；每个实例有主站发的令牌，别人无法冒充。社区库缓存在本地，主站暂时不可达时继续可用。各实例**不需要公网 IP**（都是主动连主站），但要能访问主站：主站目前只有 IPv6 地址，Linux 一键安装直接用主机网络即可；Docker 默认网桥没有 IPv6 出口，需按 `docker-compose.yml` 末尾的说明开启（Docker 27+）。系统设置 → 外部数据源 可看连接状态、社区库规模与全球已部署 / 在线 / 离线数量；开关默认打开，关闭后不再上报也不再使用社区库。自建私有社区：一台实例设 `SHARE_HUB=true`，其它实例把 `SHARE_HUB_URL` 指向它（主站要能被各实例访问，建议 HTTPS）。 4. 以上都没有时，才归到 HTTPS / 国内网络这类笼统分类。 识别库有变化或过了零点，自动把今天以前的历史流量按新规则重算（只升级不降级）；识别规则页也可手动执行。世界地图使用 [Natural Earth](https://www.naturalearthdata.com/)（公有领域）简化数据。 @@ -204,8 +204,9 @@ EOF | `SINGBOX_URL` / `SINGBOX_SECRET` / `PROXY_IPS` | 空 | 旁路代理 sing-box 的 Clash API（也可在网页填写，网页优先） | | `MOSDNS_URL` | 空 | MosDNS 审计 API 地址 | | `ADGUARD_URL` / `ADGUARD_USERNAME` / `ADGUARD_PASSWORD` | 空 | 自有 DNS 服务器 AdGuard Home 的查询日志（双层 NAT 场景） | -| `SHARE_URL` | 空 | 社区共享库中转地址（debug16）：上报本网络实测、拉取社区库；留空不参与 | -| `SHARE_TOKEN` | 空 | 中转的上报令牌（中转设置了才需要） | +| `SHARE_HUB_URL` | `https://rosdpi.suyijun.top:8443` | 社区主站地址（内置，一般不用改）；自建私有社区时改成自己的主站 | +| `SHARE_HUB` | `false` | 本实例作为社区主站（只有主站部署设为 `true`） | +| `SHARE_VERIFY_TLS` | `true` | 校验主站证书；私有社区用自签证书时才设 `false` | | `FAKEIP_CIDRS` | `198.18.0.0/15,fc00::/18` | FakeIP 地址段 | | `ASN_DB` / `CHINA_DB` / `IPLIST_DB` / `BM7_DB` / `HAGEZI_DB` / `UT1_DB` / `OUI_DB` | `true` | 各识别库自动下载开关；没有外网时设 `false` | | `WAN_INTERFACE` | 自动 | WAN 接口名，默认按默认路由识别 | @@ -252,7 +253,7 @@ backend/.venv/bin/python scripts/simulate.py --minutes 5 - **Flows**: live connections (sing-box connections show their policy group and outbound node), top destinations with a flow map from your location to each country, and proxy routing per sing-box policy group and node. - **Family members / Employees**: pick the Home or Business scenario under **Settings → Scenario & network**; group each person's devices and see their traffic, online time, top apps and behaviour tags for the last 7 days. - **Audit**: access log (CSV export), audit rules (app, domain, time window, traffic, large upload, rogue router, new device) with 14 push-notification channels, daily / weekly reports, operation log, TTL-based shared-access check, and a RouterOS control-command generator. -- **Self-learning & community library**: mappings observed on YOUR network (IP → app, domain → app) are kept as long-term memory and classify traffic even after DNS caches expire, with DoH clients, or for bare-IP flows; deployments can share anonymous votes through a relay, and the relay publishes mappings seen by at least two instances plus a global deployment counter. +- **Self-learning & community library**: mappings observed on YOUR network (IP → app, domain → app) are kept as long-term memory and classify traffic even after DNS caches expire, with DoH clients, or for bare-IP flows; deployments share anonymous votes through the built-in community hub (no third-party storage); a mapping is published only when instances on at least two different networks agree, and the hub shows global deployed / online / offline counts. Instances need no public IP — they only connect out to the hub. - **Self-update** (Linux binary install): a one-line installer sets up a systemd service with SQLite out of the box; new versions are detected automatically and applied with one click from the UI — no SSH needed. - Chinese and English UI, light and dark themes, works on phones. diff --git a/backend/app/__init__.py b/backend/app/__init__.py index 5c0d124..0a932f9 100644 --- a/backend/app/__init__.py +++ b/backend/app/__init__.py @@ -1 +1 @@ -__version__ = "0.6.20" +__version__ = "0.6.21" diff --git a/backend/app/config.py b/backend/app/config.py index 790e951..67f3134 100644 --- a/backend/app/config.py +++ b/backend/app/config.py @@ -42,9 +42,10 @@ class Settings(BaseSettings): adguard_username: str = "" # AdGuard Home 登录用户名；和密码都留空表示免认证（只在内网监听时） adguard_password: str = "" share_enabled: bool = True # 共享识别库总开关（debug16 第 3 条，默认打开）：关掉后不心跳、不上报、不拉取 - webdav_url: str = "" # 社区共享库的 WebDAV 地址（debug22，NAS 自带）：如 http://192.168.8.10:5005/ros-dpi - webdav_user: str = "" # WebDAV 账号；和密码都留空表示免认证 - webdav_password: str = "" + # 社区共享库（debug16→debug22 重做）：项目自带的社区主站，不依赖第三方存储 + share_hub_url: str = "https://rosdpi.suyijun.top:8443" # 社区主站地址；自建私有社区时改成自己的主站 + share_hub: bool = False # 本实例就是社区主站（只有主站部署设为 true）：接收上报、发布社区库与全球部署量 + share_verify_tls: bool = True # 校验主站证书；私有社区用自签证书时才设 false oui_db: bool = True # 自动下载 IEEE 的 MAC 厂商库（OUI，识别终端类型用，每 30 天更新） asn_db: bool = True # 自动下载 iptoasn 的 IP→ASN 库（公有领域，每天更新） china_db: bool = True # 自动下载国内域名列表（dnsmasq-china-list）与国内 IP 段（每天更新） diff --git a/backend/app/integrations.py b/backend/app/integrations.py index 68fc218..9fbe823 100644 --- a/backend/app/integrations.py +++ b/backend/app/integrations.py @@ -19,7 +19,7 @@ import httpx from app.config import get_settings KEYS = ("singbox_url", "singbox_secret", "proxy_ips", "mosdns_url", "adguard_url", "adguard_username", - "adguard_password", "webdav_url", "webdav_user", "webdav_password") + "adguard_password") def effective(db_values: dict) -> dict: diff --git a/backend/app/main.py b/backend/app/main.py index 7dd4576..ec6cc13 100644 --- a/backend/app/main.py +++ b/backend/app/main.py @@ -17,6 +17,7 @@ from app.models import Setting from app.ros_poller import poller from app import audit # noqa: F401 挂上审计（engine.audit） from app.routers import admin, auth, setup, stats +from app.routers import share as share_router from app.routers import audit as audit_router from app.routers import compliance as compliance_router from app.routers import people as people_router @@ -48,7 +49,7 @@ def _on_db_ready(): （0.6.0 / 0.6.1 的主域名回填出错时，后面的外部数据源配置、自定义规则都没加载，外部数据源显示“未启用”）""" ensure_admin() # 建管理员失败要报出来（安装向导靠这个提示） for name, fn in (("load_state", engine.load_state), ("apply_settings", _apply_saved), - ("local_db", _refresh_local), ("share_lib", _share_cycle)): + ("local_db", _refresh_local), ("share_lib", _share_startup)): try: fn() except Exception as e: # noqa: BLE001 @@ -76,23 +77,24 @@ def _auto_update_on() -> bool: def _share_cycle(): - """社区共享库（debug22）：心跳（部署量统计）+ 投票共享，走 NAS 的 WebDAV。开关关闭或未配置地址时跳过""" + """社区共享库：普通实例向社区主站心跳、上报、拉库；主站在进程内聚合（见 app/share.py）""" if not database.is_configured(): return from app import share with database.session() as db: - if not share.sharing_on(db): - return - saved = {k: (db.get(Setting, k).value if db.get(Setting, k) else None) for k in integrations.KEYS} - cfg = integrations.effective(saved) - url = cfg.get("webdav_url") or "" - if not url: - return - with database.session() as db: - res = share.webdav_cycle(engine, db, url, cfg.get("webdav_user") or "", cfg.get("webdav_password") or "") + res = share.cycle(engine, db) db.commit() - print(f"[share] 共享库已同步：已部署 {res['deployed']} 台（在线 {res['online']}），" - f"社区库 {res['community_ips'] + res['community_domains']} 条", flush=True) + if res: + print(f"[share] 社区共享已同步{'（本实例是社区主站）' if share.is_hub() else ''}：已部署 {res['total']} 台" + f"（在线 {res['online']}），社区库 {res['community_ips'] + res['community_domains']} 条", flush=True) + + +def _share_startup(): + """启动时先用上次缓存的社区库，再在后台跑第一轮同步（不挡住启动）""" + from app import share + if not share.is_hub(): + share.load_cached_library() + share.kick(_safe(_share_cycle, "share")) def _auto_update_rules(): @@ -210,7 +212,7 @@ async def _unhandled(request: Request, exc: Exception): for r in (setup.router, auth.router, stats.router, admin.router, audit_router.router, people_router.router, - compliance_router.router): + compliance_router.router, share_router.router): app.include_router(r) diff --git a/backend/app/models.py b/backend/app/models.py index 4d170f5..4b942cc 100644 --- a/backend/app/models.py +++ b/backend/app/models.py @@ -227,13 +227,17 @@ class ShareEntry(Base): class ShareDeployment(Base): - """社区部署心跳（debug16）：实例每小时上报一次存在，中转聚合出 全球已部署 / 在线 / 离线。只含版本号，无任何网络数据""" + """社区主站上的实例登记（debug16 / 0.6.21 重做）：注册时发令牌，每次上报更新心跳。 + 全球 已部署 / 在线 / 离线 从这里统计；只含版本号和来源网络的哈希，不存来源 IP""" __tablename__ = "share_deployments" instance = Column(String(36), primary_key=True) version = Column(String(16)) first_seen = Column(BigInteger, nullable=False) last_seen = Column(BigInteger, nullable=False) - public_url = Column(String(255)) # 中转实例的公开地址（部署量里的可用中转列表用） + public_url = Column(String(255)) # 0.6.15~0.6.19 的中转地址，已停用 + token_hash = Column(String(64)) # 实例令牌的 SHA-256（令牌本身只在实例那边） + net = Column(String(16)) # 来源网络（IPv4 /24、IPv6 /56）的加盐哈希：防一人伪造多个实例刷票 + last_votes = Column(BigInteger) # 最近一次带投票的上报时间 class ShareDomain(Base): diff --git a/backend/app/ros_poller.py b/backend/app/ros_poller.py index 3c62a8e..538bbe3 100644 --- a/backend/app/ros_poller.py +++ b/backend/app/ros_poller.py @@ -9,6 +9,16 @@ import httpx from app.config import get_settings + +def _global_v6(addr: str | None) -> bool: + """RouterOS /ipv6/address 的 address（带前缀长度）是不是公网地址""" + import ipaddress + try: + a = ipaddress.ip_interface(addr or "").ip + except ValueError: + return False + return a.version == 6 and a.is_global + class RosPoller: def __init__(self, engine): self.engine = engine @@ -32,6 +42,8 @@ class RosPoller: self.samples: deque = deque(maxlen=1440) self.mangle: list[dict] = [] # 接入自检用：RouterOS 的 mangle 规则（只读） self.ip_settings: dict = {} + self.filter_rules: list[dict] | None = None # /ip/firewall/filter（看 FastTrack 规则有没有限定 connection-bytes） + self.has_ipv6 = False # 路由器上有没有 IPv6 公网地址 self._sample_prev: dict[str, tuple[float, int, int]] = {} @property @@ -85,12 +97,16 @@ class RosPoller: extra = {k: self._opt(c, path) for k, path in ( ("pools", "/ip/pool"), ("servers", "/ip/dhcp-server"), ("dhcpc", "/ip/dhcp-client"), ("pppoe", "/interface/pppoe-client"), ("mangle", "/ip/firewall/mangle"), - ("mangle6", "/ipv6/firewall/mangle"), ("ipset", "/ip/settings"))} + ("mangle6", "/ipv6/firewall/mangle"), ("ipset", "/ip/settings"), + ("filter", "/ip/firewall/filter"))} self._apply(ifaces, arp, leases, addrs, routes, nb6, addrs6) self._apply_status(ifaces, addrs, leases, routes, extra) self.mangle = [{**r, "_family": "ipv4"} for r in extra["mangle"] or []] + \ [{**r, "_family": "ipv6"} for r in extra["mangle6"] or []] self.ip_settings = extra["ipset"] if isinstance(extra["ipset"], dict) else {} + self.filter_rules = extra["filter"] if isinstance(extra["filter"], list) else None + self.has_ipv6 = any(_global_v6(a.get("address")) and a.get("disabled") != "true" + for a in addrs6 or []) # 全部成功后再更新展示用的状态，失败时保留上一次的结果 self.identity = ident.get("name", "") if isinstance(ident, dict) else "" self.resource = resource if isinstance(resource, dict) else {} @@ -280,10 +296,19 @@ class RosPoller: issues.append("no_sni") elif not any(x["kind"] == "sni" and x["protocol"] == "udp" for x in rules): issues.append("no_quic") - if self.ip_settings.get("ipv4-fasttrack-active") == "true": + # 有 IPv6 公网地址却只有 IPv4 的嗅探规则：走 IPv6 的 DNS 应答和握手全都看不到（国内运营商默认下发 IPv6） + if self.has_ipv6 and any(x["family"] == "ipv4" for x in rules) and not any(x["family"] == "ipv6" and not x["disabled"] for x in rules): + issues.append("no_ipv6") + fasttrack = self.ip_settings.get("ipv4-fasttrack-active") == "true" + # FastTrack 规则已经加了 connection-bytes（连接开头几 KB 不走 FastTrack）时握手照样镜像得到，不算问题； + # 读不到 filter 表（权限不够时是空列表）时按旧办法提示 + ft_rules = [r for r in (self.filter_rules or []) if r.get("action") == "fasttrack-connection" + and r.get("disabled") != "true"] + ft_limited = bool(ft_rules) and all(r.get("connection-bytes") for r in ft_rules) + if fasttrack and not ft_limited: issues.append("fasttrack") return {"checked": True, "port": port, "target": ref[0] if ref else None, "rules": rules, "issues": issues, - "fasttrack": self.ip_settings.get("ipv4-fasttrack-active") == "true"} + "fasttrack": fasttrack, "fasttrack_limited": fasttrack and ft_limited} def status(self) -> dict: r = self.resource if isinstance(self.resource, dict) else {} diff --git a/backend/app/routers/admin.py b/backend/app/routers/admin.py index f923aa9..a8aec59 100644 --- a/backend/app/routers/admin.py +++ b/backend/app/routers/admin.py @@ -132,9 +132,6 @@ class SettingsIn(BaseModel): adguard_url: str | None = Field(None, max_length=255) adguard_username: str | None = Field(None, max_length=255) adguard_password: str | None = Field(None, max_length=255) - webdav_url: str | None = Field(None, max_length=255) - webdav_user: str | None = Field(None, max_length=255) - webdav_password: str | None = Field(None, max_length=255) rules_auto_update: bool | None = None share_enabled: bool | None = None @@ -179,8 +176,6 @@ def _integrations(db): "mosdns_url": cfg["mosdns_url"], "adguard_url": cfg["adguard_url"], "adguard_username": cfg["adguard_username"], "has_adguard_password": bool(cfg["adguard_password"]), - "webdav_url": cfg["webdav_url"], "webdav_user": cfg["webdav_user"], - "has_webdav_password": bool(cfg["webdav_password"]), "proxy_auto": integrations.singbox.host_ip()} @@ -229,6 +224,9 @@ def put_settings(req: SettingsIn, request: Request, user: str = Depends(current_ _put(db, "rules_auto_update", "1" if req.rules_auto_update else "0") if req.share_enabled is not None: _put(db, "share_enabled", "1" if req.share_enabled else "0") + db.commit() + from app import share + share.kick(share.run_once) if req.wan_interface is not None: _put(db, "wan_interface", req.wan_interface.strip()) poller.wan_override = req.wan_interface.strip() @@ -311,8 +309,9 @@ def library(db: Session = Depends(get_db)): @router.get("/system/deployments") def system_deployments(db: Session = Depends(get_db)): - """全球部署量 + 可用中转列表（debug19）""" - return deployments_with_relays(db) + """社区共享状态 + 全球部署量（设置页 → 外部数据源 → 社区共享库）""" + from app import share + return share.status(db) @router.get("/system/update") diff --git a/backend/app/routers/share.py b/backend/app/routers/share.py new file mode 100644 index 0000000..0294224 --- /dev/null +++ b/backend/app/routers/share.py @@ -0,0 +1,89 @@ +"""社区主站的公开端点（0.6.21 重做，见 app/share.py）。只有 SHARE_HUB=true 的实例开放，其它实例一律 404。 + +- POST /api/share/v1/register 首次注册，领取实例 ID 和令牌 +- POST /api/share/v1/report 心跳 + 投票（Authorization: Bearer <实例ID>:<令牌>，可 gzip） +- GET /api/share/v1/library 社区库（ETag / 304，支持 gzip） +- GET /api/share/v1/stats 全球部署量：已部署 / 在线 / 离线（所有人可见） +""" +from fastapi import APIRouter, HTTPException, Request, Response +from fastapi.responses import JSONResponse +from starlette.concurrency import run_in_threadpool + +from app import database, share + +router = APIRouter(prefix=share.API, tags=["share"]) + + +def _need_hub(): + if not share.is_hub(): + raise HTTPException(404, "本实例不是社区主站") + if not database.is_configured(): + raise HTTPException(503, "社区主站的数据库还没就绪") + + +def _source(request: Request) -> str: + return share.client_ip(request.client.host if request.client else "", request.headers.get("x-forwarded-for"), + request.headers.get("x-real-ip")) + + +def _run(fn): + try: + return fn() + except share.ShareError as e: + raise HTTPException(e.status, e.detail) from None + + +@router.post("/register") +async def register(request: Request): + _need_hub() + if int(request.headers.get("content-length") or 0) > share.MAX_BODY: + raise HTTPException(413, "请求内容太大") + raw = await request.body() + ip = _source(request) + + def work(): + body = share.inflate(raw, request.headers.get("content-encoding")) + with database.session() as db: + iid, token = share.register(db, share.net_of(db, ip), str(body.get("version") or "")) + db.commit() + return {"instance": iid, "token": token} + return await run_in_threadpool(_run, work) + + +@router.post("/report") +async def report(request: Request): + _need_hub() + if int(request.headers.get("content-length") or 0) > share.MAX_BODY: + raise HTTPException(413, "上报内容太大") + raw = await request.body() + ip = _source(request) + + def work(): + from app.engine import engine + body = share.inflate(raw, request.headers.get("content-encoding")) + with database.session() as db: + row = share.authenticate(db, request.headers.get("authorization")) + res = share.report(db, row, share.net_of(db, ip), body, engine.classifier) + db.commit() + return res + return await run_in_threadpool(_run, work) + + +@router.get("/library") +def library(request: Request): + _need_hub() + with database.session() as db: + lib = share.library(db) + headers = {"ETag": lib["etag"], "Cache-Control": "no-cache"} + if request.headers.get("if-none-match") == lib["etag"]: + return Response(status_code=304, headers=headers) + if "gzip" in (request.headers.get("accept-encoding") or "").lower(): + return Response(lib["gzip"], media_type="application/json", headers={**headers, "Content-Encoding": "gzip"}) + return Response(lib["json"], media_type="application/json", headers=headers) + + +@router.get("/stats") +def stats(): + _need_hub() + with database.session() as db: + return JSONResponse(share.stats(db), headers={"Access-Control-Allow-Origin": "*"}) diff --git a/backend/app/share.py b/backend/app/share.py index 4b5bd5a..274ced2 100644 --- a/backend/app/share.py +++ b/backend/app/share.py @@ -1,48 +1,76 @@ -"""社区共享库（debug16→debug22）：通过 **WebDAV 文件夹**（飞牛 / 群晖等 NAS 自带）共享本地自学习库。 +"""社区共享库（debug16 → 0.6.21 重做）：各部署实例互相共享本地自学习库的实测结论，项目自带社区主站，不依赖第三方存储。 -架构（无中心服务、无中转）： -- 每个实例把各自的匿名投票写到 WebDAV 上的两个文件（文件名是随机 UUID，不含任何可识别信息）： - deployments/.json {"version", "ts"} —— 部署心跳（全球部署量统计） - votes/.json {"entries": [IP→应用], "domains": [域名→应用]} -- 每 30 分钟：PUT 自己的两个文件（WebDAV 天然按文件去重，不会互相覆盖），然后列出文件夹里 - 所有其它实例的文件、全部拉下来在本地聚合——**至少 2 个不同实例见过的映射**才进入社区层。 -- 本地实测优先：社区条目只补本地没有结论的 IP / 域名。 +架构： +- 社区主站：和普通实例是同一套程序，部署时设 SHARE_HUB=true（只有主站这样设）。它接收各实例的上报， + 聚合后发布社区库与全球部署量（/api/share/v1/*，见 routers/share.py）。主站自己的投票在进程内直接计入， + 不经过网络（之前的中转要通过公网地址访问自己，NAT 回环不通时永远同步不上）。 +- 普通实例：默认参与共享，主站地址内置（SHARE_HUB_URL 可改成自建的私有社区）。每 30 分钟一轮： + 首次注册领取令牌 → 上报心跳（每 6 小时附带一次投票）→ 拉取社区库（没变化时主站回 304）→ 拉取部署量。 + 社区库缓存在 DATA_DIR/share_community.json，主站暂时不可达时继续用上次的版本。 -隐私与防污染： -- 投票内容只有 (远端 IP / 域名, 应用 ID, 字节票数)；不含本机终端、域名记录、自定义应用； -- 私网 / 保留 / 组播地址剔除；IPv6 按 /64 前缀聚合（中国环境以 IPv6 公网为主）； -- 应用白名单：只共享所有实例都认识的应用，自定义应用和编造的 ID 拒收； -- 单文件限量，防止单实例撑爆共享文件夹。 +隐私： +- 投票只有 (远端 IP / 域名, 应用 ID, 字节票数)；不含本机终端、访问记录、自定义应用； +- 私网 / 保留 / 组播地址剔除；IPv6 按 /64 前缀聚合； +- 主站不保存来源 IP，只保存来源网络（IPv4 /24、IPv6 /56）的加盐哈希。 -安全：WebDAV 地址建议用 HTTPS 或仅内网可达；账号密码保存在本实例数据库里，不上报。 +防污染： +- 每个实例有主站发的令牌，别人不能冒充或覆盖它的投票；同一来源网络每天限注册次数；投票每小时最多收一次； +- 一条映射要有 ≥2 个**不同来源网络**的实例投给同一个应用，并且票数多于其它应用，才进社区库—— + 一个人伪造再多实例 ID，来自同一个网络也只算一票； +- 应用白名单（自定义应用、编造的 ID 拒收）、域名格式校验、单实例条数上限、字节票数上限。 """ +import gzip +import hashlib +import hmac import ipaddress import json +import os import re +import secrets import threading import time -import urllib.parse import uuid from collections import defaultdict import httpx -from sqlalchemy import select +from sqlalchemy import delete, func, select from app.classify.local import _is_ip, _v6key -from app.models import HostDomainDay +from app.config import get_settings +from app.models import HostDomainDay, Setting, ShareDeployment, ShareDomain, ShareEntry -MAX_UPLOAD = 50_000 # 单个实例投票文件的条数上限 -MAX_COMMUNITY = 200_000 # 社区聚合后的条数上限 -PUBLISH_FILES = 2 # 至少几个不同实例的投票文件见过才进社区层 -DEPLOY_ONLINE = 7200 # 心跳 2 小时内算在线 +API = "/api/share/v1" +MAX_UPLOAD = 50_000 # 单次上报 IP / 域名各自的条数上限 +MAX_COMMUNITY = 200_000 # 社区库 IP / 域名各自的条数上限 +PUBLISH_NETS = 2 # 至少几个不同来源网络投给同一应用才发布 +DEPLOY_ONLINE = 7200 # 2 小时内有心跳算在线 +DEPLOY_KEEP = 90 * 86400 # 90 天没心跳的实例不再计入部署量 +ENTRY_TTL = 30 * 86400 # 30 天没再上报投票的实例，投票作废 +VOTES_EVERY = 6 * 3600 # 客户端每 6 小时附带一次投票（心跳每轮都发） +VOTES_MIN_GAP = 3600 # 主站对同一实例的投票每小时最多收一次 +REGISTER_PER_NET = 5 # 同一来源网络每天最多注册几个实例 +MAX_BODY = 8 * 1024 * 1024 # 上报请求体上限（压缩后） +MAX_INFLATED = 48 * 1024 * 1024 # 解压后上限，防压缩炸弹 +LIB_REBUILD_GAP = 60 # 社区库最快每分钟重建一次 +MAX_BYTES = 10 ** 15 # 单条字节票数上限 +HUB_NET = "hub" # 主站自己的投票的来源网络标记 -# 域名格式（debug20：格式错误的域名条目直接丢弃） +# 域名格式：格式错误的域名条目直接丢弃 _DOMAIN_RE = re.compile(r"^(?=.{1,253}$)([a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?\.)+[a-z]{2,63}$") -MAX_BYTES = 10 ** 15 + + +class ShareError(Exception): + def __init__(self, status: int, detail: str): + super().__init__(detail) + self.status, self.detail = status, detail + + +# ---------- 共用 ---------- _sync_lock = threading.Lock() _sync = {"ts": 0, "ok": None, "error": "", "community_ips": 0, "community_domains": 0, - "deployed": 0, "online": 0, "offline": 0} + "total": None, "online": None, "offline": None} +_client = {"votes_ts": 0, "etag": ""} def sync_state() -> dict: @@ -50,19 +78,38 @@ def sync_state() -> dict: return dict(_sync) -def instance_id(db) -> str: - """本实例的匿名 ID：第一次生成后存在 settings 表（不上报任何可识别信息）""" - from app.models import Setting - row = db.get(Setting, "share_instance_id") - if row is not None and row.value: - return row.value - iid = str(uuid.uuid4()) - row = db.get(Setting, "share_instance_id") +def _set_sync(**kw): + with _sync_lock: + _sync.update(kw) + + +def is_hub() -> bool: + return bool(get_settings().share_hub) + + +def hub_url() -> str: + return (get_settings().share_hub_url or "").strip().rstrip("/") + + +def sharing_on(db) -> bool: + """共享识别库的总开关（debug16 第 3 条）：默认打开；关闭后不心跳、不上报、不拉取（主站不受影响，照常服务其它实例）""" + row = db.get(Setting, "share_enabled") + if row is not None and row.value is not None: + return row.value == "1" + return bool(get_settings().share_enabled) + + +def _setting(db, key: str) -> str: + row = db.get(Setting, key) + return row.value if row is not None and row.value else "" + + +def _put_setting(db, key: str, value: str): + row = db.get(Setting, key) if row is None: - db.add(Setting(key="share_instance_id", value=iid)) + db.add(Setting(key=key, value=value)) else: - row.value = iid - return iid + row.value = value def _is_shareable_app(classifier, app: str) -> bool: @@ -80,166 +127,446 @@ def _shareable_ip(ip: str) -> bool: return not (a.is_private or a.is_reserved or a.is_multicast or a.is_loopback or a.is_link_local) -def export_entries(engine, db) -> list[dict]: - """本地自学习库的 IP 条目：host_domain_day 里以 IP 记录的行，按字节数取最多的应用。 - IPv4 按精确地址；IPv6 按 /64 前缀聚合（中国环境以 IPv6 公网为主，精确地址不稳定）""" +def export_entries(engine, db) -> tuple[list[dict], list[dict]]: + """本地自学习库整理成投票：host_domain_day 里的行按字节数取最多的应用。 + IP 条目 IPv4 按精确地址、IPv6 按 /64 前缀（精确地址在不同网络间几乎不重复，凑不满投票）；域名条目按精确域名""" classifier = engine.classifier - votes: dict = defaultdict(dict) - rows = db.execute(select(HostDomainDay.domain, HostDomainDay.app, HostDomainDay.bytes)).all() - for dom, app, b in rows: - if not _shareable_ip(dom) or not _is_shareable_app(classifier, app): - continue - key = _v6key(dom) or dom - per = votes[key] - per[app] = per.get(app, 0) + int(b or 0) - out = [{"ip": ip, "app": max(per, key=per.get), "bytes": sum(per.values())} - for ip, per in votes.items()] - out.sort(key=lambda x: -x["bytes"]) - doms = export_domains(engine, db) - return out[:MAX_UPLOAD], doms - - -def export_domains(engine, db) -> list[dict]: - """本地自学习库的域名条目（debug17）：非 IP 的域名行，按字节数取最多的应用，精确域名共享""" - classifier = engine.classifier - votes: dict = defaultdict(dict) - rows = db.execute(select(HostDomainDay.domain, HostDomainDay.app, HostDomainDay.bytes)).all() - for dom, app, b in rows: - if _is_ip(dom) or not _is_shareable_app(classifier, app): - continue - per = votes[dom] - per[app] = per.get(app, 0) + int(b or 0) - out = [{"domain": d, "app": max(p, key=p.get), "bytes": sum(p.values())} - for d, p in votes.items()] - out.sort(key=lambda x: -x["bytes"]) - return out[:MAX_UPLOAD] - - -# ---------- WebDAV ---------- - -def _dav_url(base: str, path: str) -> str: - return base.rstrip("/") + "/" + "/".join(urllib.parse.quote(x) for x in path.split("/")) - - -def _dav_put(httpx, base: str, path: str, data, auth) -> bool: - """PUT 文件；目标文件夹不存在（409）时逐级 MKCOL 后重试一次""" - url = _dav_url(base, path) - r = httpx.put(url, content=data, auth=auth, timeout=30.0, verify=False) - if r.status_code in (200, 201, 204): - return True - if r.status_code == 409: # 上级文件夹不存在：逐级创建 - parts = path.split("/") - for i in range(1, len(parts)): - httpx.request("MKCOL", _dav_url(base, "/".join(parts[:i])), auth=auth, - timeout=30.0, verify=False) - r = httpx.put(url, content=data, auth=auth, timeout=30.0, verify=False) - return r.status_code in (200, 201, 204) - return False - - -def _dav_list(httpx, base: str, folder: str, auth) -> list[str]: - """PROPFIND 列出文件夹里的 .json 文件，返回 folder/name 相对路径（和 _dav_get 拼接一致）""" - r = httpx.request("PROPFIND", _dav_url(base, folder), headers={"Depth": "1"}, auth=auth, - timeout=30.0, verify=False) - if r.status_code not in (200, 207): - return [] - hrefs = re.findall(r"<(?:[\w]+:)?href>\s*([^<]+)\s*", r.text, re.IGNORECASE) - base_path = urllib.parse.quote(_dav_url(base, "").split("//", 1)[-1].split("/", 1)[-1]) - out = [] - for h in hrefs: - name = urllib.parse.unquote(h.split("?")[0].rstrip("/")).rsplit("/", 1)[-1] - if name.endswith(".json") and name != folder: - out.append(f"{folder}/{name}") - return out - - -def _dav_get(httpx, base: str, path: str, auth): - r = httpx.get(_dav_url(base, path), auth=auth, timeout=30.0, verify=False) - if r.status_code != 200: - return None - try: - return json.loads(r.text) - except Exception: # noqa: BLE001 - return None - - -def webdav_cycle(engine, db, base: str, user: str, password: str) -> dict: - """一轮 WebDAV 共享（每 30 分钟）：写自己的心跳与投票文件 → 读其它实例的文件 → 本地聚合""" - from app import __version__ - auth = (user, password) if user or password else None - iid = instance_id(db) - now = int(time.time()) - - # 1) 心跳（部署量统计：只有版本号和时间） - ok_put_dep = _dav_put(httpx, base, f"deployments/{iid}.json", - json.dumps({"version": __version__, "ts": now}).encode(), auth) - # 2) 投票（本网络实测） - entries, domains = export_entries(engine, db) - ok_put_votes = _dav_put(httpx, base, f"votes/{iid}.json", - json.dumps({"version": __version__, "ts": now, - "entries": entries, "domains": domains}, - ensure_ascii=False).encode(), auth) - - # 3) 读其它实例的文件，本地聚合（自己写不进去时留给下轮，不空算） - other_dep, other_votes = [], [] - if ok_put_dep and ok_put_votes: - for kind, out in (("deployments", other_dep), ("votes", other_votes)): - for path in _dav_list(httpx, base, kind, auth): - if path == f"{kind}/{iid}.json": - continue - data = _dav_get(httpx, base, path, auth) - if isinstance(data, dict): - out.append(data) - - # 部署量：心跳文件数 = 已部署；ts 在 2 小时内 = 在线 - deployed, online = len(other_dep) + (1 if ok_put_dep else 0), 0 - for d in other_dep: - if int(d.get("ts", 0) or 0) >= now - DEPLOY_ONLINE: - online += 1 - if ok_put_dep: - online += 1 - - # 社区层：出现在 ≥2 个不同投票文件里的映射（自己的文件算一票）。votes[k] = {app: (实例数, 总字节)} - def _vote(bucket, key, app, weight): - per = bucket[key] - n, b = per.get(app, (0, 0)) - per[app] = (n + 1, b + weight) - ip_votes: dict = defaultdict(dict) dom_votes: dict = defaultdict(dict) - if ok_put_votes: - for e in entries: - _vote(ip_votes, e["ip"], e["app"], e["bytes"]) - for e in domains: - _vote(dom_votes, e["domain"], e["app"], e["bytes"]) - for d in other_votes: - for e in d.get("entries") or []: - ip, app, b = str(e.get("ip", "")), str(e.get("app", "")), int(e.get("bytes", 0) or 0) - if _shareable_ip(ip): - _vote(ip_votes, ip, app, b) - for e in d.get("domains") or []: - dom_, app, b = str(e.get("domain", "")).lower(), str(e.get("app", "")), int(e.get("bytes", 0) or 0) - if dom_ and not _is_ip(dom_): - _vote(dom_votes, dom_, app, b) + rows = db.execute(select(HostDomainDay.domain, HostDomainDay.app, HostDomainDay.bytes)).all() + for dom, app, b in rows: + if not dom or not _is_shareable_app(classifier, app): + continue + if _is_ip(dom): + if not _shareable_ip(dom): + continue + per = ip_votes[_v6key(dom) or dom] + else: + per = dom_votes[dom.lower().rstrip(".")] + per[app] = per.get(app, 0) + int(b or 0) - def top(bucket, cap): - out = {} - for key, per in sorted(bucket.items(), key=lambda kv: -sum(v[1] for v in kv[1].values()))[:cap]: - if any(n >= PUBLISH_FILES for n, _b in per.values()): # 至少 2 个实例见过 - out[key] = max(per, key=lambda a: per[a][1]) - return out + def pack(votes, field): + out = [{field: k, "app": max(p, key=p.get), "bytes": sum(p.values())} for k, p in votes.items()] + out.sort(key=lambda x: -x["bytes"]) + return out[:MAX_UPLOAD] + return pack(ip_votes, "ip"), pack(dom_votes, "domain") - community_ips = top(ip_votes, MAX_COMMUNITY) - community_domains = top(dom_votes, MAX_COMMUNITY) + +def apply_library(data: dict) -> tuple[int, int]: + """把社区库并入本地自学习库的社区层（本地实测优先，社区只补空白）。返回 (IP 条数, 域名条数)""" from app.classify.local import local - local.load_community(community_ips, community_domains) + ips, doms = {}, {} + for e in (data or {}).get("entries") or []: + if not isinstance(e, dict): + continue + ip, app = str(e.get("ip", "")), str(e.get("app", "")) + if app and len(app) <= 48 and _shareable_ip(ip[:-3] if ip.endswith("/64") else ip): + ips[ip] = app + for e in (data or {}).get("domains") or []: + if not isinstance(e, dict): + continue + dom, app = str(e.get("domain", "")).lower().rstrip("."), str(e.get("app", "")) + if app and len(app) <= 48 and _DOMAIN_RE.match(dom): + doms[dom] = app + local.load_community(ips, doms) + return len(ips), len(doms) - ok = ok_put_dep and ok_put_votes - error = "" if ok else "WebDAV 写入失败：检查地址 / 账号密码 / 文件夹权限" - with _sync_lock: - _sync.update(ts=now, ok=ok, error=error, deployed=deployed, online=online, - offline=deployed - online, - community_ips=len(community_ips), community_domains=len(community_domains)) - return {"deployed": deployed, "online": online, "offline": deployed - online, - "community_ips": len(community_ips), "community_domains": len(community_domains), - "uploaded": bool(ok_put_dep and ok_put_votes)} + +def _cache_path() -> str: + return os.path.join(os.environ.get("DATA_DIR", "data"), "share_community.json") + + +def load_cached_library(): + """启动时先用上次拉到的社区库，不用等第一轮同步；主站暂时不可达时也一直有数据""" + try: + with open(_cache_path(), encoding="utf-8") as f: + data = json.load(f) + except (OSError, ValueError): + return + n_ip, n_dom = apply_library(data) + _client["etag"] = str(data.get("etag") or "") + _set_sync(community_ips=n_ip, community_domains=n_dom) + + +def _save_cache(data: dict, etag: str): + try: + os.makedirs(os.path.dirname(_cache_path()) or ".", exist_ok=True) + tmp = _cache_path() + ".tmp" + with open(tmp, "w", encoding="utf-8") as f: + json.dump({**data, "etag": etag}, f, ensure_ascii=False) + os.replace(tmp, _cache_path()) + except OSError as e: + print(f"[share] 社区库缓存写入失败：{e}", flush=True) + + +# ---------- 社区主站 ---------- + +def net_of(db, ip: str) -> str: + """来源网络的加盐哈希（IPv4 /24、IPv6 /56）：同一网络里伪造多个实例也只算一票；主站不保存来源 IP 本身。 + IPv6 按 /56：国内家庭宽带一般分到 /56 或 /60，按 /48 会把几百户人家算成同一个网络""" + try: + a = ipaddress.ip_address(ip) + except ValueError: + return "unknown" + net = ipaddress.ip_network(f"{a}/{24 if a.version == 4 else 56}", strict=False) + salt = _setting(db, "share_salt") + if not salt: + salt = secrets.token_hex(16) + _put_setting(db, "share_salt", salt) + return hashlib.sha256(f"{salt}|{net}".encode()).hexdigest()[:16] + + +def client_ip(peer: str, forwarded: str | None, real_ip: str | None) -> str: + """主站看到的来源地址。直接暴露在公网时用 TCP 对端；对端是内网（前面有反向代理）时，取代理追加在 + X-Forwarded-For 最右边的公网地址——直连的人伪造这个头没用，因为那时对端是公网地址、头会被忽略""" + try: + p = ipaddress.ip_address(peer) + behind_proxy = p.is_private or p.is_loopback + except ValueError: + behind_proxy = False + if not behind_proxy: + return peer + for cand in reversed([x.strip() for x in (forwarded or "").split(",") if x.strip()]): + if _shareable_ip(cand): + return cand + if real_ip and _shareable_ip(real_ip.strip()): + return real_ip.strip() + return peer + + +def _hash_token(token: str) -> str: + return hashlib.sha256(token.encode()).hexdigest() + + +def register(db, net: str, version: str) -> tuple[str, str]: + """主站：登记一个新实例并发放令牌（调用方提交事务）。同一来源网络每天限 REGISTER_PER_NET 个""" + now = int(time.time()) + recent = db.scalar(select(func.count()).select_from(ShareDeployment) + .where(ShareDeployment.net == net, ShareDeployment.first_seen >= now - 86400)) or 0 + if net != HUB_NET and recent >= REGISTER_PER_NET: + raise ShareError(429, "注册太频繁，请明天再试") + iid, token = str(uuid.uuid4()), secrets.token_urlsafe(32) + db.add(ShareDeployment(instance=iid, version=version[:16], first_seen=now, last_seen=now, + token_hash=_hash_token(token), net=net)) + return iid, token + + +def authenticate(db, auth_header: str | None) -> ShareDeployment: + """主站：校验 Authorization: Bearer <实例ID>:<令牌>""" + raw = (auth_header or "").removeprefix("Bearer ").strip() + iid, _, token = raw.partition(":") + row = db.get(ShareDeployment, iid[:36]) if iid and token else None + if row is None or not row.token_hash or not hmac.compare_digest(row.token_hash, _hash_token(token)): + raise ShareError(401, "实例未注册或令牌无效") + return row + + +def _clamp_bytes(v) -> int: + try: + return min(max(int(v or 0), 0), MAX_BYTES) + except (TypeError, ValueError): + return 0 + + +def report(db, row: ShareDeployment, net: str, body: dict, classifier) -> dict: + """主站：记录一次上报（调用方提交事务）。心跳每次都记；带了投票时整份替换该实例的旧投票。 + 应用白名单：只收主站也认识的内置应用，自定义应用和编造的 ID 拒收""" + now = int(time.time()) + row.last_seen, row.net = now, net + row.version = str(body.get("version") or "")[:16] + entries, domains = body.get("entries"), body.get("domains") + if not isinstance(entries, list) and not isinstance(domains, list): + return {"ok": True, "accepted": 0} + if row.last_votes and now - row.last_votes < VOTES_MIN_GAP: + return {"ok": True, "accepted": 0, "skipped": "投票每小时最多收一次"} + ips: dict = {} + for e in (entries or [])[:MAX_UPLOAD]: + if not isinstance(e, dict): + continue + ip, app = str(e.get("ip", "")), str(e.get("app", "")) + base = ip[:-3] if ip.endswith("/64") else ip + if not _shareable_ip(base) or not _is_shareable_app(classifier, app): + continue + ips[_v6key(base) or base] = (app, _clamp_bytes(e.get("bytes"))) + doms: dict = {} + for e in (domains or [])[:MAX_UPLOAD]: + if not isinstance(e, dict): + continue + dom, app = str(e.get("domain", "")).lower().rstrip("."), str(e.get("app", "")) + if not _DOMAIN_RE.match(dom) or not _is_shareable_app(classifier, app): + continue + doms[dom] = (app, _clamp_bytes(e.get("bytes"))) + db.execute(delete(ShareEntry).where(ShareEntry.reporter == row.instance)) + db.execute(delete(ShareDomain).where(ShareDomain.reporter == row.instance)) + db.add_all([ShareEntry(ip=k, app=a, reporter=row.instance, bytes=b, ts=now) for k, (a, b) in ips.items()]) + db.add_all([ShareDomain(domain=k, app=a, reporter=row.instance, bytes=b, ts=now) for k, (a, b) in doms.items()]) + row.last_votes = now + _lib["dirty"] = True + return {"ok": True, "accepted": len(ips) + len(doms)} + + +def purge(db): + """主站：清掉 90 天没心跳的实例、30 天没更新的投票（调用方提交事务）""" + now = int(time.time()) + db.execute(delete(ShareDeployment).where(ShareDeployment.last_seen < now - DEPLOY_KEEP)) + n = db.execute(delete(ShareEntry).where(ShareEntry.ts < now - ENTRY_TTL)).rowcount or 0 + n += db.execute(delete(ShareDomain).where(ShareDomain.ts < now - ENTRY_TTL)).rowcount or 0 + if n: + _lib["dirty"] = True + + +def _aggregate(rows, nets: dict) -> dict: + """按来源网络计票：每个键选投票网络最多的应用（平票看字节），网络数 ≥ PUBLISH_NETS 且多于第二名才发布""" + votes: dict = defaultdict(lambda: defaultdict(lambda: [set(), 0])) + for key, app, reporter, b in rows: + net = nets.get(reporter) + if net is None: # 已被清理的实例、0.6.20 以前的旧中转留下的投票不算 + continue + v = votes[key][app] + v[0].add(net) + v[1] += int(b or 0) + out = [] + for key, per in votes.items(): + ranked = sorted(per.items(), key=lambda kv: (len(kv[1][0]), kv[1][1]), reverse=True) + top_app, (top_nets, top_bytes) = ranked[0] + runner = len(ranked[1][1][0]) if len(ranked) > 1 else 0 + if len(top_nets) >= PUBLISH_NETS and len(top_nets) > runner: + out.append((top_bytes, key, top_app)) + out.sort(reverse=True) + return {key: app for _b, key, app in out[:MAX_COMMUNITY]} + + +def build_library(db) -> dict: + nets = {i: n for i, n in db.execute(select(ShareDeployment.instance, ShareDeployment.net)).all() if n} + ips = _aggregate(db.execute(select(ShareEntry.ip, ShareEntry.app, ShareEntry.reporter, ShareEntry.bytes)).all(), nets) + doms = _aggregate(db.execute(select(ShareDomain.domain, ShareDomain.app, ShareDomain.reporter, + ShareDomain.bytes)).all(), nets) + return {"version": 1, "generated": int(time.time()), + "entries": [{"ip": k, "app": v} for k, v in sorted(ips.items())], + "domains": [{"domain": k, "app": v} for k, v in sorted(doms.items())]} + + +_lib_lock = threading.Lock() +_lib = {"dirty": True, "ts": 0, "etag": "", "json": b"", "gzip": b"", "data": None} + + +def library(db) -> dict: + """主站：当前社区库（有新投票时最快每分钟重建一次），返回 {etag, json, gzip, data}""" + with _lib_lock: + if _lib["data"] is None or (_lib["dirty"] and time.time() - _lib["ts"] >= LIB_REBUILD_GAP): + data = build_library(db) + raw = json.dumps({k: data[k] for k in ("version", "entries", "domains")}, ensure_ascii=False, + separators=(",", ":")).encode() + body = json.dumps(data, ensure_ascii=False, separators=(",", ":")).encode() + _lib.update(dirty=False, ts=time.time(), etag='"' + hashlib.sha256(raw).hexdigest()[:32] + '"', + json=body, gzip=gzip.compress(body, 6), data=data) + return dict(_lib) + + +def reset_library_cache(): + """测试用""" + with _lib_lock: + _lib.update(dirty=True, ts=0, etag="", json=b"", gzip=b"", data=None) + _client.update(votes_ts=0, etag="") + + +def stats(db) -> dict: + """全球部署量（公开）：已部署 = 90 天内有心跳的实例，在线 = 2 小时内有心跳""" + now = int(time.time()) + # 只数 0.6.21 起注册过的实例：旧中转（0.6.10~0.6.19）留下的无令牌心跳不计，否则升级后会重复计数 + q = select(func.count()).select_from(ShareDeployment).where(ShareDeployment.token_hash.is_not(None)) + total = db.scalar(q.where(ShareDeployment.last_seen >= now - DEPLOY_KEEP)) or 0 + online = db.scalar(q.where(ShareDeployment.last_seen >= now - DEPLOY_ONLINE)) or 0 + return {"total": total, "online": online, "offline": total - online} + + +def inflate(body: bytes, encoding: str | None) -> dict: + """主站：解析上报请求体（可 gzip 压缩），限制解压后大小""" + if len(body) > MAX_BODY: + raise ShareError(413, "上报内容太大") + if (encoding or "").lower() == "gzip": + import zlib + d = zlib.decompressobj(16 + zlib.MAX_WBITS) + try: + out = d.decompress(body, MAX_INFLATED) + except zlib.error: + raise ShareError(400, "上报内容解压失败") from None + if d.unconsumed_tail: + raise ShareError(413, "上报内容太大") + body = out + try: + data = json.loads(body or b"{}") + except ValueError: + raise ShareError(400, "上报内容不是 JSON") from None + if not isinstance(data, dict): + raise ShareError(400, "上报内容格式不对") + return data + + +def hub_cycle(engine, db) -> dict: + """主站自己的一轮：投票在进程内直接计入（不经网络）→ 清理过期 → 重建社区库并用到本机""" + from app import __version__ + iid = _setting(db, "share_hub_instance") + row = db.get(ShareDeployment, iid) if iid else None + if row is None: + iid, _token = register(db, HUB_NET, __version__) + _put_setting(db, "share_hub_instance", iid) + db.flush() + row = db.get(ShareDeployment, iid) + body: dict = {"version": __version__} + on = sharing_on(db) + if on and (not row.last_votes or time.time() - row.last_votes >= VOTES_EVERY): + body["entries"], body["domains"] = export_entries(engine, db) + elif not on: + # 主站关掉“参与共享”：照常服务其它实例，只是不再贡献自己的投票、本机也不用社区库 + db.execute(delete(ShareEntry).where(ShareEntry.reporter == iid)) + db.execute(delete(ShareDomain).where(ShareDomain.reporter == iid)) + row.last_votes = None + _lib["dirty"] = True + report(db, row, HUB_NET, body, engine.classifier) + purge(db) + db.flush() + lib = library(db) + if on: + n_ip, n_dom = apply_library(lib["data"]) + else: + from app.classify.local import local + local.load_community({}, {}) + n_ip = n_dom = 0 + st = stats(db) + _set_sync(ts=int(time.time()), ok=True, error="", community_ips=n_ip, community_domains=n_dom, **st) + return {**st, "community_ips": n_ip, "community_domains": n_dom} + + +# ---------- 普通实例 ---------- + +def _http() -> httpx.Client: + from app import __version__ + return httpx.Client(verify=get_settings().share_verify_tls, timeout=30.0, follow_redirects=False, + headers={"User-Agent": f"ros-dpi/{__version__}"}) + + +def _err(e: Exception) -> str: + if isinstance(e, httpx.HTTPStatusError): + try: + detail = e.response.json().get("detail") or "" + except ValueError: + detail = "" + return f"主站返回 {e.response.status_code}" + (f"：{detail}" if detail else "") + if isinstance(e, httpx.ConnectError) and "CERTIFICATE" in str(e).upper(): + return "主站证书校验失败（私有社区用自签证书时设 SHARE_VERIFY_TLS=false）" + if isinstance(e, ShareError): + return e.detail + msg = str(e).lower() + if isinstance(e, httpx.ConnectError) and ("errno 101" in msg or "unreachable" in msg + or "cannot assign requested address" in msg): + # 实例不需要公网 IP（都是主动连主站），但主站目前只有 IPv6 地址：本机 / 容器要有 IPv6 出口 + return "连不上社区主站：本机没有 IPv6 出口（主站只有 IPv6 地址）。Docker 部署请按 docker-compose.yml 末尾的说明开启 IPv6，或改用 Linux 一键安装" + return f"{type(e).__name__}: {str(e)[:120]}" + + +def _register(c: httpx.Client, base: str, db) -> tuple[str, str]: + from app import __version__ + r = c.post(base + API + "/register", json={"version": __version__}) + r.raise_for_status() + d = r.json() + iid, token = str(d["instance"]), str(d["token"]) + _put_setting(db, "share_client_instance", iid) + _put_setting(db, "share_client_token", token) + _put_setting(db, "share_client_hub", base) + db.commit() + return iid, token + + +def client_cycle(engine, db) -> dict: + """普通实例一轮同步：注册（首次）→ 心跳（每 6 小时带投票）→ 拉社区库 → 拉部署量""" + from app import __version__ + base = hub_url() + if not base: + raise ShareError(400, "没有配置社区主站地址（SHARE_HUB_URL）") + with _http() as c: + iid, token = _setting(db, "share_client_instance"), _setting(db, "share_client_token") + if not iid or not token or _setting(db, "share_client_hub") != base: # 换了主站要重新注册 + iid, token = _register(c, base, db) + _client["votes_ts"] = 0 + body: dict = {"version": __version__} + with_votes = time.time() - _client["votes_ts"] >= VOTES_EVERY + if with_votes: + body["entries"], body["domains"] = export_entries(engine, db) + payload = gzip.compress(json.dumps(body, ensure_ascii=False, separators=(",", ":")).encode(), 6) + + def send(i, t): + return c.post(base + API + "/report", content=payload, + headers={"Authorization": f"Bearer {i}:{t}", "Content-Type": "application/json", + "Content-Encoding": "gzip"}) + r = send(iid, token) + if r.status_code == 401: # 主站清理过（90 天没心跳）或换过库：重新注册一次 + iid, token = _register(c, base, db) + r = send(iid, token) + r.raise_for_status() + if with_votes: + _client["votes_ts"] = time.time() + + headers = {"If-None-Match": _client["etag"]} if _client["etag"] else {} + r = c.get(base + API + "/library", headers=headers) + if r.status_code == 304: + st = sync_state() + n_ip, n_dom = st["community_ips"], st["community_domains"] + else: + r.raise_for_status() + data = r.json() + n_ip, n_dom = apply_library(data) + _client["etag"] = r.headers.get("etag", "") + _save_cache(data, _client["etag"]) + r = c.get(base + API + "/stats") + r.raise_for_status() + st = {k: int(r.json().get(k) or 0) for k in ("total", "online", "offline")} + _set_sync(ts=int(time.time()), ok=True, error="", community_ips=n_ip, community_domains=n_dom, **st) + return {**st, "community_ips": n_ip, "community_domains": n_dom} + + +def cycle(engine, db) -> dict | None: + """定时任务入口（每 30 分钟）：主站走进程内流程；普通实例开关关闭时什么都不发""" + if is_hub(): + return hub_cycle(engine, db) + if not sharing_on(db): + from app.classify.local import local + local.load_community({}, {}) + _set_sync(ts=0, ok=None, error="", community_ips=0, community_domains=0, + total=None, online=None, offline=None) + return None + try: + return client_cycle(engine, db) + except Exception as e: # noqa: BLE001 记录原因给设置页显示；社区库继续用上次缓存 + _set_sync(ts=int(time.time()), ok=False, error=_err(e)) + raise + + +def kick(fn): + """后台立刻跑一轮（启动后、打开开关后），不等 30 分钟的定时任务；测试环境不跑""" + if not os.environ.get("ROSDPI_NO_JOBS"): + threading.Thread(target=fn, daemon=True).start() + + +def run_once(): + from app import database + from app.engine import engine + try: + with database.session() as db: + cycle(engine, db) + db.commit() + except Exception as e: # noqa: BLE001 + print(f"[share] 同步失败：{_err(e)}", flush=True) + + +def status(db) -> dict: + """设置页的共享状态：角色、主站地址、开关、上次同步结果与全球部署量""" + st = sync_state() + if is_hub(): # 主站的部署量直接查库，不用等下一轮 + st.update(stats(db)) + # 不同来源网络数：实例很多却只有 1 个网络，说明反向代理没有传 X-Forwarded-For，社区库永远凑不够票 + st["nets"] = db.scalar(select(func.count(func.distinct(ShareDeployment.net))) + .where(ShareDeployment.token_hash.is_not(None), + ShareDeployment.net != HUB_NET)) or 0 + return {"role": "hub" if is_hub() else "client", "hub_url": hub_url(), "enabled": sharing_on(db), + "verify_tls": bool(get_settings().share_verify_tls), "sync": st} diff --git a/backend/tests/test_debug16.py b/backend/tests/test_debug16.py index 65e4d71..88da25f 100644 --- a/backend/tests/test_debug16.py +++ b/backend/tests/test_debug16.py @@ -1,152 +1,254 @@ -"""debug16/22：WebDAV 社区共享——本地库导出过滤、≥2 实例聚合、部署量、自学习接入。""" +"""社区共享库（debug16 → 0.6.21 重做）：项目自带社区主站——注册 / 令牌、心跳 + 投票、按来源网络聚合防刷票、 +ETag 304、部署量；普通实例的完整同步流程走真实的主站端点（TestClient），不打桩协议本身。""" +import gzip import json -import time import pytest +from fastapi.testclient import TestClient +from app import database, share from app.classify.local import local -from app.engine import Engine -from app.models import HostDomainDay -from app.share import webdav_cycle +from app.config import get_settings +from app.engine import engine +from app.main import app +from app.models import HostDomainDay, Setting, ShareDeployment, ShareDomain, ShareEntry +from app.routers import share as share_router DAY = 1792900000 -HOST = "IP:192.168.8.40" -BASE = "http://nas.local:5005/ros-dpi" +HOST = "IP:192.168.88.41" +HUB = "http://testserver" -def _engine(): - e = Engine() - e.set_lan("192.168.8.0/24") - return e +def _clean(db): + for m in (ShareEntry, ShareDomain, ShareDeployment): + db.query(m).delete(synchronize_session=False) + db.query(HostDomainDay).filter(HostDomainDay.host == HOST).delete(synchronize_session=False) + for k in ("share_client_instance", "share_client_token", "share_client_hub", "share_hub_instance"): + row = db.get(Setting, k) + if row is not None: + db.delete(row) + db.commit() -class _FakeDav: - """最小 WebDAV 仿真：按文件夹分桶的内存键值 + PROPFIND 列表；记录所有 PUT。 - files 的键是 "/.json"（和真实 WebDAV 的路径一致）""" +@pytest.fixture() +def hub(monkeypatch, tmp_path): + """本进程同时当社区主站和普通实例：主站端点由 TestClient 提供；X-Test-IP 头模拟不同实例的来源地址""" + s = get_settings() + monkeypatch.setattr(s, "share_hub", True) + monkeypatch.setattr(s, "share_hub_url", HUB) + monkeypatch.setattr(share, "LIB_REBUILD_GAP", 0) + monkeypatch.setenv("DATA_DIR", str(tmp_path)) + monkeypatch.setattr(share_router, "_source", lambda req: req.headers.get("x-test-ip") or "203.0.113.9") + database.connect_from_saved(force=True) + with database.session() as db: + _clean(db) + share.reset_library_cache() + local.reset() + c = TestClient(app) + state = {"ip": "198.51.100.1"} - def __init__(self): - self.files = {} - self.puts = [] + class _Client: + """client_cycle 用的 httpx.Client 替身：请求转给 TestClient，并带上当前模拟的来源地址""" + def __enter__(self): + return self - def handle(self, method, url, content=None): - from httpx import Response - path = str(url).split("/ros-dpi/")[-1].rstrip("/") # deployments/xxx.json 或 votes/ 或 votes/xxx.json - if method == "PROPFIND": - folder = path.rstrip("/") - names = [k for k in self.files if k.startswith(folder + "/")] - body = "".join(f"/{n}" for n in names) - return Response(207, text=f"{body}") - if method == "PUT": - self.puts.append(path) - self.files[path] = content or b"" - return Response(201) - if method == "GET": - if path in self.files: - return Response(200, content=self.files[path]) - return Response(404) - if method == "MKCOL": - return Response(201) - return Response(200) + def __exit__(self, *a): + return False + + def post(self, url, **kw): + kw["headers"] = {**(kw.get("headers") or {}), "X-Test-IP": state["ip"]} + return c.post(url, **kw) + + def get(self, url, **kw): + kw["headers"] = {**(kw.get("headers") or {}), "X-Test-IP": state["ip"]} + return c.get(url, **kw) + + monkeypatch.setattr(share, "_http", lambda: _Client()) + yield c, state + with database.session() as db: + _clean(db) + share.reset_library_cache() + local.reset() -def _install(monkeypatch, dav): - """把 share.py 里的 httpx 模块函数换成 WebDAV 仿真""" - import httpx +def _as_new_instance(state, ip): + """换成另一台实例：清掉本机保存的注册信息，换来源地址""" + with database.session() as db: + for k in ("share_client_instance", "share_client_token", "share_client_hub"): + row = db.get(Setting, k) + if row is not None: + db.delete(row) + db.commit() + share._client.update(votes_ts=0, etag="") + state["ip"] = ip - from app import share as share_mod - monkeypatch.setattr(share_mod.httpx, "put", lambda url, content=None, **kw: dav.handle("PUT", url, content)) - monkeypatch.setattr(share_mod.httpx, "get", lambda url, **kw: dav.handle("GET", url)) - monkeypatch.setattr(share_mod.httpx, "request", lambda method, url, **kw: dav.handle(method, url)) - del httpx +def _seed(db, rows): + for dom, app, b in rows: + db.add(HostDomainDay(ts=DAY, host=HOST, domain=dom, app=app, base=None, bytes=b)) + db.commit() + + +def _client_cycle(): + with database.session() as db: + res = share.client_cycle(engine, db) + db.commit() + return res def test_export_filters_private(): - import app.database as database database.connect_from_saved(force=True) - s = database.SessionLocal() - try: - s.query(HostDomainDay).filter(HostDomainDay.host == HOST).delete(synchronize_session=False) - s.add(HostDomainDay(ts=DAY, host=HOST, domain="93.184.216.34", app="douyin", base=None, bytes=800)) - s.add(HostDomainDay(ts=DAY, host=HOST, domain="192.168.8.5", app="douyin", base=None, bytes=999)) # 私网 - s.add(HostDomainDay(ts=DAY, host=HOST, domain="239.255.255.250", app="douyin", base=None, bytes=999)) # 组播 - s.commit() - eng = _engine() - from app import share as share_mod - entries, _domains = share_mod.export_entries(eng, s) - ips = {e["ip"] for e in entries} - assert "93.184.216.34" in ips - assert "192.168.8.5" not in ips and "239.255.255.250" not in ips - finally: - s.query(HostDomainDay).filter(HostDomainDay.host == HOST).delete(synchronize_session=False) - s.commit() - s.close() + with database.session() as db: + _clean(db) + _seed(db, [("93.184.216.34", "douyin", 800), ("192.168.8.5", "douyin", 999), + ("239.255.255.250", "douyin", 999), ("v.douyin.com", "douyin", 50)]) + entries, domains = share.export_entries(engine, db) + _clean(db) + ips = {e["ip"] for e in entries} + assert "93.184.216.34" in ips and not ips & {"192.168.8.5", "239.255.255.250"} # 私网 / 组播不出门 + assert "v.douyin.com" in {d["domain"] for d in domains} -def test_webdav_aggregates_two_instances(monkeypatch): - """两个实例各写各的投票文件：≥2 个实例见过的映射进社区层；单实例见过的进不来""" - import app.database as database - database.connect_from_saved(force=True) - dav = _FakeDav() - _install(monkeypatch, dav) - s = database.SessionLocal() - try: - s.query(HostDomainDay).filter(HostDomainDay.host == HOST).delete(synchronize_session=False) - s.commit() - eng = _engine() - # A 的本机实测：1.2.3.4 → douyin（进入自己的投票文件） - s.add(HostDomainDay(ts=DAY, host=HOST, domain="1.2.3.4", app="douyin", base=None, bytes=500)) - s.commit() - # 第一轮：A 写入自己的投票文件；此时没有其它文件，社区层为空 - with database.session() as db: - res1 = webdav_cycle(eng, db, BASE, "", "") - db.commit() - assert res1["deployed"] == 1 and res1["online"] == 1 - assert res1["community_ips"] == 0 - # B 的文件出现在 NAS 上：1.2.3.4 → douyin（与 A 相同）+ 5.6.7.8 → bilibili（只有 B 见过） - dav.files["votes/IID_B.json"] = json.dumps({ - "version": "0.6.20", "ts": int(time.time()), - "entries": [{"ip": "1.2.3.4", "app": "douyin", "bytes": 100}, - {"ip": "5.6.7.8", "app": "bilibili", "bytes": 200}], - "domains": []}).encode() - dav.files["deployments/IID_B.json"] = json.dumps( - {"version": "0.6.20", "ts": int(time.time())}).encode() - - # 第二轮：A 读 B 的文件并聚合 - with database.session() as db: - res2 = webdav_cycle(eng, db, BASE, "", "") - db.commit() - assert res2["deployed"] == 2 and res2["online"] == 2 - assert res2["community_ips"] == 1 # 1.2.3.4 两家都见过 - assert local.ip_app("1.2.3.4") == "douyin" - assert local.ip_app("5.6.7.8") is None # 单实例见过的进不了社区层 - assert len(dav.puts) == 4 # 2 轮 × (deployments + votes) - finally: - s.query(HostDomainDay).filter(HostDomainDay.host == HOST).delete(synchronize_session=False) - s.commit() - s.close() - local.reset() - dav.files.clear() - dav.puts.clear() +def test_client_ip_behind_proxy(): + # 直连公网：用 TCP 对端，伪造的 X-Forwarded-For 无效 + assert share.client_ip("8.8.8.8", "1.1.1.1", None) == "8.8.8.8" + # 反向代理在内网：取代理追加在最右边的公网地址 + assert share.client_ip("192.168.8.1", "1.1.1.1, 9.9.9.9", None) == "9.9.9.9" + assert share.client_ip("127.0.0.1", "10.0.0.5", "9.9.9.9") == "9.9.9.9" -def test_webdav_single_instance_no_community(monkeypatch): +def test_two_networks_publish_end_to_end(hub): + c, state = hub + with database.session() as db: + _seed(db, [("1.2.3.4", "douyin", 500), ("v.douyin.com", "douyin", 300)]) + + # 第一台：注册 + 上报；只有一个网络见过，社区库为空 + res = _client_cycle() + assert res["total"] == 1 and res["online"] == 1 and res["community_ips"] == 0 + assert share.sync_state()["ok"] is True and local.ip_app("1.2.3.4") is None + + # 第二台（另一个网络）投了同样的结论：两网一致 → 进社区库，本机自学习的社区层生效 + _as_new_instance(state, "203.0.113.50") + res = _client_cycle() + assert res["total"] == 2 and res["community_ips"] >= 1 and res["community_domains"] >= 1 + assert local.ip_app("1.2.3.4") == "douyin" + assert local.domain_app("v.douyin.com") == "douyin" + + # 社区库没变化：带 ETag 再拉是 304，结果沿用；缓存文件已写，重启后可直接加载 + etag = share._client["etag"] + assert etag and c.get(HUB + share.API + "/library", headers={"If-None-Match": etag}).status_code == 304 + local.reset() + share.load_cached_library() + assert local.ip_app("1.2.3.4") == "douyin" + + # 公开的部署量 + assert c.get(HUB + share.API + "/stats").json() == {"total": 2, "online": 2, "offline": 0} + + +def test_same_network_counts_once(hub): + """一个人在同一网络里伪造多个实例：只算一票，进不了社区库""" + c, state = hub + with database.session() as db: + _seed(db, [("5.6.7.8", "bilibili", 100)]) + _client_cycle() + _as_new_instance(state, "198.51.100.77") # 同一个 /24 + res = _client_cycle() + assert res["total"] == 2 and res["community_ips"] == 0 and local.ip_app("5.6.7.8") is None + + +def test_majority_and_whitelist(hub): + c, _state = hub + with database.session() as db: + rows = [] + for i, ip in enumerate(("1.0.0.1", "2.0.0.1", "3.0.0.1")): + iid, _t = share.register(db, share.net_of(db, ip), "t") + db.flush() + rows.append(db.get(ShareDeployment, iid)) + cls = engine.classifier + share.report(db, rows[0], rows[0].net, {"entries": [{"ip": "8.8.8.8", "app": "douyin", "bytes": 1}]}, cls) + share.report(db, rows[1], rows[1].net, {"entries": [{"ip": "8.8.8.8", "app": "douyin", "bytes": 1}]}, cls) + share.report(db, rows[2], rows[2].net, {"entries": [{"ip": "8.8.8.8", "app": "bilibili", "bytes": 10 ** 9}, + {"ip": "9.9.9.9", "app": "made_up_app", "bytes": 1}, + {"ip": "10.0.0.1", "app": "douyin", "bytes": 1}]}, cls) + db.commit() + lib = share.build_library(db) + n = db.query(ShareEntry).filter(ShareEntry.reporter == rows[2].instance).count() + assert lib["entries"] == [{"ip": "8.8.8.8", "app": "douyin"}] # 两个网络对一个网络：按网络数，不按字节 + assert n == 1 # 编造的应用、私网地址被拒收 + + +def test_auth_and_limits(hub): + c, _state = hub + url = HUB + share.API + assert c.post(url + "/report", json={"version": "x"}).status_code == 401 + reg = c.post(url + "/register", json={"version": "x"}, headers={"X-Test-IP": "192.0.2.1"}).json() + bad = {"Authorization": f"Bearer {reg['instance']}:wrong"} + assert c.post(url + "/report", json={"version": "x"}, headers=bad).status_code == 401 + ok = {"Authorization": f"Bearer {reg['instance']}:{reg['token']}", "Content-Encoding": "gzip"} + assert c.post(url + "/report", content=gzip.compress(json.dumps({"version": "x"}).encode()), headers=ok).json()["ok"] + # 解压后超限：拒收（防压缩炸弹） + bomb = gzip.compress(b" " * (share.MAX_INFLATED + 10)) + assert c.post(url + "/report", content=bomb, headers=ok).status_code == 413 + # 同一网络一天注册超限 + for i in range(share.REGISTER_PER_NET): + assert c.post(url + "/register", json={}, headers={"X-Test-IP": f"100.20.30.{i}"}).status_code == 200 + assert c.post(url + "/register", json={}, headers={"X-Test-IP": "100.20.30.200"}).status_code == 429 + + +def test_hub_counts_itself_in_process(hub): + """主站自己的投票在进程内计入（不经网络）：主站 + 一个普通实例两网一致即发布""" + c, state = hub + with database.session() as db: + _seed(db, [("4.4.4.4", "douyin", 100)]) + res = share.hub_cycle(engine, db) + db.commit() + assert res["total"] == 1 and res["community_ips"] == 0 + assert local.ip_app("4.4.4.4") is None + _client_cycle() + with database.session() as db: + share._lib["dirty"] = True + res = share.hub_cycle(engine, db) + db.commit() + assert res["total"] == 2 and res["community_ips"] >= 1 and local.ip_app("4.4.4.4") == "douyin" + + +def test_non_hub_404(monkeypatch): + monkeypatch.setattr(get_settings(), "share_hub", False) + c = TestClient(app) + for path in ("/stats", "/library"): + assert c.get(share.API + path).status_code == 404 + assert c.post(share.API + "/register", json={}).status_code == 404 + + +def test_client_error_is_reported(monkeypatch): + """主站不可达：设置页能看到原因，社区库沿用缓存""" import httpx - dav = _FakeDav() - _install(monkeypatch, dav) - eng = _engine() - try: - local.reset() - import app.database as database - database.connect_from_saved(force=True) - s = database.SessionLocal() - with database.session() as db: - res = webdav_cycle(eng, db, BASE, "", "") - db.commit() - assert res["deployed"] == 1 and res["online"] == 1 - assert res["community_ips"] == 0 # 没有其它实例 → 社区层空（本地实测本来就优先） - finally: - s.close() - local.reset() - dav.files.clear() + database.connect_from_saved(force=True) + monkeypatch.setattr(get_settings(), "share_hub", False) + monkeypatch.setattr(get_settings(), "share_hub_url", "http://127.0.0.1:9") + + class _Down: + def __enter__(self): + return self + + def __exit__(self, *a): + return False + + def post(self, url, **kw): + raise httpx.ConnectError("connection refused") + get = post + + monkeypatch.setattr(share, "_http", lambda: _Down()) + with database.session() as db: + with pytest.raises(httpx.ConnectError): + share.cycle(engine, db) + st = share.sync_state() + assert st["ok"] is False and "ConnectError" in st["error"] + + +def test_no_ipv6_egress_message(): + """实例不需要公网 IP，但主站只有 IPv6：容器没有 IPv6 出口时给出能照着做的提示""" + import httpx + msg = share._err(httpx.ConnectError("[Errno 101] Network is unreachable")) + assert "IPv6" in msg and "Docker" in msg diff --git a/backend/tests/test_ros_poller.py b/backend/tests/test_ros_poller.py index 053e8f9..4cb8f89 100644 --- a/backend/tests/test_ros_poller.py +++ b/backend/tests/test_ros_poller.py @@ -92,10 +92,32 @@ def test_poll_success_populates_router_state(monkeypatch): assert chk["checked"] and sni["problems"] == ["port"] and sni["target_port"] == 37880 # SNI 规则存在、只是端口错：建议改端口，不建议重复新增；但缺 UDP（QUIC）规则 assert "rule_problem" in chk["issues"] and "no_sni" not in chk["issues"] and "no_quic" in chk["issues"] + # 路由器有 IPv6 公网地址，嗅探规则却只有 IPv4 的一份 + assert "no_ipv6" in chk["issues"] and "fasttrack" not in chk["issues"] assert not codes["rules"]["ok"] and codes["rules"]["params"]["wrong"] == [37880] leases = c.get("/api/dhcp/leases", headers={"Authorization": f"Bearer {tok}"}).json() assert leases["leases"][0]["mac"] == "AA:BB:CC:DD:EE:31" and leases["leases"][0]["host_name"] == "iPhone" +def test_fasttrack_check_respects_connection_bytes(monkeypatch): + """FastTrack 开着：规则没限定 connection-bytes 要提示；已经限定（连接开头不走 FastTrack）就不再提示""" + _use_mock(monkeypatch) + monkeypatch.setitem(engine.stats, "dns_answers", 5) + monkeypatch.setitem(ROS, "/rest/ip/settings", {"ipv4-fasttrack-active": "true"}) + rule = {".id": "*2", "action": "fasttrack-connection", "chain": "forward", + "connection-state": "established,related", "disabled": "false"} + monkeypatch.setitem(ROS, "/rest/ip/firewall/filter", [rule]) + poller.poll() + assert "fasttrack" in poller.check()["issues"] + monkeypatch.setitem(ROS, "/rest/ip/firewall/filter", [{**rule, "connection-bytes": "10000-0"}]) + poller.poll() + chk = poller.check() + assert "fasttrack" not in chk["issues"] and chk["fasttrack_limited"] + # 读不到 filter 表（只读账号没权限）：按旧办法提示 + monkeypatch.delitem(ROS, "/rest/ip/firewall/filter") + poller.poll() + assert "fasttrack" in poller.check()["issues"] + + def test_poll_401_gives_actionable_error_and_keeps_status_usable(monkeypatch): _use_mock(monkeypatch, status=401) poller.poll() diff --git a/backend/tests/test_routes_smoke.py b/backend/tests/test_routes_smoke.py new file mode 100644 index 0000000..659a57a --- /dev/null +++ b/backend/tests/test_routes_smoke.py @@ -0,0 +1,28 @@ +"""所有无路径参数的 GET 接口都不能返回 500（0.6.20 的部署量接口调用了已删除的函数，上线后才发现）""" +from fastapi.routing import APIRoute +from fastapi.testclient import TestClient + +from app.main import app + +SKIP = {"/api/system/update"} # 会访问 GitHub + + +def _h(c): + if not c.get("/api/auth/status").json()["initialized"]: + c.post("/api/auth/setup", json={"username": "admin", "password": "secret1"}) + return {"Authorization": "Bearer " + c.post("/api/auth/login", json={"username": "admin", "password": "secret1"}).json()["token"]} + + +def test_get_routes_no_500(): + with TestClient(app) as c: + h = _h(c) + bad = [] + for r in app.routes: + if not isinstance(r, APIRoute) or "GET" not in r.methods or "{" in r.path: + continue + if not r.path.startswith("/api/") or r.path in SKIP: + continue + resp = c.get(r.path, headers=h) + if resp.status_code >= 500: + bad.append((r.path, resp.status_code, resp.text[:120])) + assert not bad, bad diff --git a/docker-compose.with-mysql.yml b/docker-compose.with-mysql.yml index aa0a31c..120ce33 100644 --- a/docker-compose.with-mysql.yml +++ b/docker-compose.with-mysql.yml @@ -53,3 +53,10 @@ services: interval: 10s timeout: 5s retries: 20 + +# 社区共享库：社区主站目前只有 IPv6 地址，而 Docker 默认网桥没有 IPv6 出口，容器会连不上主站 +# （设置页显示“连接失败：本机没有 IPv6 出口”）。Docker 27 及以上取消下面 3 行注释即可（自动分配地址并做 NAT）； +# 更老的 Docker（群晖 / 部分 NAS 自带的版本）请改用 Linux 一键安装，或在 daemon.json 里开启 ipv6 + ip6tables。 +# networks: +# default: +# enable_ipv6: true diff --git a/docker-compose.yml b/docker-compose.yml index 78b2f3c..2170988 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -26,3 +26,10 @@ services: # 旁路代理（sing-box）和内网 DNS（MosDNS）在网页 系统设置 → 外部数据源 里填写，不需要写在这里 volumes: - ./data:/app/data # 保存数据库连接配置，务必保留这个卷 + +# 社区共享库：社区主站目前只有 IPv6 地址，而 Docker 默认网桥没有 IPv6 出口，容器会连不上主站 +# （设置页显示“连接失败：本机没有 IPv6 出口”）。Docker 27 及以上取消下面 3 行注释即可（自动分配地址并做 NAT）； +# 更老的 Docker（群晖 / 部分 NAS 自带的版本）请改用 Linux 一键安装，或在 daemon.json 里开启 ipv6 + ip6tables。 +# networks: +# default: +# enable_ipv6: true diff --git a/frontend/src/i18n/en.js b/frontend/src/i18n/en.js index 1196e1d..de0d126 100644 --- a/frontend/src/i18n/en.js +++ b/frontend/src/i18n/en.js @@ -155,13 +155,13 @@ export default { agUrl: 'AdGuard Home address', agUser: 'Login username (leave empty if none)', agPassword: 'Login password', agKept: 'Set; leave empty to keep', agNone: 'Leave empty if none', agClear: 'Clear password', agHint: 'AdGuard Home query log must be on (enabled by default); empty username and password mean unauthenticated access.', - shareTitle: 'Community library (WebDAV)', - deploys: 'Global deployments', relaysLabel: 'Available relays', deployed: '{n} deployed', online: '{n} online', offline: '{n} offline', deployHint: 'from the relay, refreshed hourly', - shareEnabled: 'Share the learned library (on by default)', shareNoUrl: 'No relay address set', shareWait: 'Configured, waiting for the first sync', + shareTitle: 'Community library', + deploys: 'Global deployments', deployed: '{n} deployed', online: '{n} online', offline: '{n} offline', deployHint: 'from the community hub, refreshed every 30 minutes', + shareEnabled: 'Join community sharing (on by default)', shareWait: 'Waiting for the first sync', shareHub: 'This instance is the community hub', + shareNets: 'Source networks', shareNetsWarn: 'Several instances but only 1 source network: the reverse proxy in front of the hub does not pass X-Forwarded-For, so every instance looks like the same network and nothing reaches the vote threshold. Enable forwarding the client IP in the proxy.', + shareHubUrl: 'Community hub', shareHubHint: 'built in, nothing to fill in; set SHARE_HUB_URL to run a private community', communitySize: 'Community library', communityDetail: '{ip} IP memories · {dom} domain memories', - shareWhy: 'Every deployment learns different IP→app mappings. Pooling anonymous votes makes the shared identification sharper for everyone. Reports contain remote-IP→app votes only — no local devices, no domain records, no custom apps; private ranges are stripped.', - shareUrl: 'WebDAV folder URL', shareUser: 'WebDAV username', sharePassword: 'WebDAV password', - shareHint: 'Enable WebDAV on your NAS and create a folder (e.g. ros-dpi), then put its URL here. Every 30 minutes: your anonymous votes are written, votes of other instances are read and aggregated locally (entries seen by 2+ instances win). HTTPS or LAN-only recommended.', + shareWhy: 'Every deployment learns different IP/domain→app mappings. Anonymous votes are pooled on the community hub; a mapping is published only when instances on at least 2 different networks agree. Reports contain no local devices, no access records, no custom apps; private ranges are stripped. Turning this off stops reporting and stops using the community library.', asn: 'IP ownership (ASN)', asnOn: 'On', asnWhy: 'Domain-less traffic is attributed to the owning company by IP (Tencent, Alibaba, Apple, Netflix…). Data from iptoasn.com (public domain), refreshed daily; off without internet access.', asnRows: '{n} ranges, updated {d}.', connected: 'Connected', failed: 'Failed', off: 'Off', @@ -177,7 +177,8 @@ export default { issue: { no_dns: 'No working DNS sniffing rule.', no_sni: 'No working SNI rule: HTTPS / QUIC handshakes never reach ROS-DPI.', no_quic: 'Only a TCP SNI rule; UDP 443 (QUIC / HTTP3) is missing, so QUIC-heavy apps such as Douyin and YouTube are not identified.', - fasttrack: 'FastTrack is on: fast-tracked connections skip mangle, so handshakes are not mirrored. Add connection-bytes=10000-0 to the FastTrack rule.', + fasttrack: 'FastTrack is on and its rule has no connection-bytes limit: fast-tracked connections skip mangle, so handshakes are not mirrored and much HTTPS falls back to IP-only buckets. The command below keeps the first 10 KB of each connection off FastTrack; CPU cost barely changes.', + no_ipv6: 'The router has public IPv6 addresses but sniff rules exist for IPv4 only: DNS answers and HTTPS / QUIC handshakes over IPv6 are invisible. The commands below add the same rules under /ipv6 firewall mangle; if sniff-target is rejected, use the IPv6 address of ROS-DPI.', }, }, audit: { diff --git a/frontend/src/i18n/zh.js b/frontend/src/i18n/zh.js index 1032dfe..1fe9e6a 100644 --- a/frontend/src/i18n/zh.js +++ b/frontend/src/i18n/zh.js @@ -155,13 +155,13 @@ export default { agUrl: 'AdGuard Home 地址', agUser: '登录用户名（没有可留空）', agPassword: '登录密码', agKept: '已设置，不修改请留空', agNone: '没有设置可留空', agClear: '清除密码', agHint: '需要 AdGuard Home 开启查询日志（默认开启）；账号和密码都留空表示免认证访问。', - shareTitle: '社区共享库（WebDAV）', - deploys: '全球部署量', relaysLabel: '可用中转', deployed: '已部署 {n}', online: '在线 {n}', offline: '离线 {n}', deployHint: '来自社区中转，每小时刷新', - shareEnabled: '共享识别库（默认打开）', shareNoUrl: '未配置中转地址', shareWait: '已配置，等待首次同步', + shareTitle: '社区共享库', + deploys: '全球部署量', deployed: '已部署 {n}', online: '在线 {n}', offline: '离线 {n}', deployHint: '来自社区主站，每 30 分钟刷新', + shareEnabled: '参与社区共享（默认打开）', shareWait: '等待首次同步', shareHub: '本实例是社区主站', + shareNets: '来源网络', shareNetsWarn: '多个实例却只看到 1 个来源网络：主站前面的反向代理没有传 X-Forwarded-For，所有实例被当成同一网络，社区库凑不够票。请在反向代理里开启传递客户端 IP。', + shareHubUrl: '社区主站', shareHubHint: '项目内置，不需要填写；自建私有社区时用环境变量 SHARE_HUB_URL 修改', communitySize: '社区库现有', communityDetail: 'IP 记忆 {ip} 条 · 域名记忆 {dom} 条', - shareWhy: '每个部署实例的自学习库都不一样：把各自实测的「远端 IP → 应用」匿名投票汇总到一起，大家共同的识别就更准。上报内容不含本机终端、域名记录与自定义应用，私网地址自动剔除。', - shareUrl: 'WebDAV 共享文件夹地址', shareUser: 'WebDAV 账号', sharePassword: 'WebDAV 密码', - shareHint: '在 NAS 上开启 WebDAV 并建一个共享文件夹（如 ros-dpi），把地址填到这里；每 30 分钟同步一次：写入自己的匿名投票、读取其它实例的投票并在本地聚合（至少 2 台见过才采用）。飞牛 / 群晖在应用中心开启 WebDAV 即可；建议 HTTPS 或仅内网使用。', + shareWhy: '每个部署实例的自学习库都不一样：把各自实测的「远端 IP / 域名 → 应用」匿名投票汇总到社区主站，至少 2 个不同网络的实例都这样认的才进入社区库，大家的识别就更准。上报内容不含本机终端、访问记录与自定义应用，私网地址自动剔除；关闭后不再上报，也不再使用社区库。', asn: 'IP 归属库（ASN）', asnOn: '已启用', asnWhy: '没有域名的流量按 IP 归属认出公司（腾讯、阿里、苹果、Netflix……）。数据来自 iptoasn.com（公有领域），容器每天自动更新；没有外网时不启用。', asnRows: '共 {n} 条，更新于 {d}。', connected: '已连接', failed: '连接失败', off: '未启用', @@ -177,7 +177,8 @@ export default { issue: { no_dns: '没有生效的 DNS 嗅探规则。', no_sni: '没有生效的 SNI 规则：HTTPS / QUIC 握手到不了 ROS-DPI。', no_quic: '只有 TCP 的 SNI 规则，缺少 UDP 443（QUIC / HTTP3），抖音、YouTube 等大量走 QUIC 的流量认不出来。', - fasttrack: 'FastTrack 已启用：被 FastTrack 的连接不经过 mangle，握手包镜像不到。可以给 FastTrack 规则加 connection-bytes=10000-0。', + fasttrack: 'FastTrack 已启用，且规则没有限定 connection-bytes：被 FastTrack 的连接不经过 mangle，握手包镜像不到，大量 HTTPS 只能按 IP 归到「国内网络」。下方命令让每条连接的前 10KB 不走 FastTrack，CPU 占用几乎不变。', + no_ipv6: '路由器有 IPv6 公网地址，但嗅探规则只有 IPv4 的：走 IPv6 的 DNS 应答和 HTTPS / QUIC 握手都看不到（国内运营商默认下发 IPv6，这部分流量通常很大）。下方命令在 /ipv6 firewall mangle 补一份；若提示 sniff-target 无效，改填 ROS-DPI 的 IPv6 地址。', }, }, audit: { diff --git a/frontend/src/views/Rules.vue b/frontend/src/views/Rules.vue index 4ae7a1c..30f261c 100644 --- a/frontend/src/views/Rules.vue +++ b/frontend/src/views/Rules.vue @@ -263,7 +263,8 @@ async function reclassify() { reclassing.value = true reclassRes.value = '' try { - const r = await http.post('/rules/reclassify') + // 数据多时要一两分钟（审计报告 P1-3：实测 80 秒），默认 20 秒超时会误报失败而后端其实还在跑 + const r = await http.post('/rules/reclassify', null, { timeout: 600000 }) reclassRes.value = t('rules.reclassifyDone', { b: bytes(r.bytes_moved), s: r.seconds }) } catch (e) { reclassRes.value = e.message } finally { reclassing.value = false } } diff --git a/frontend/src/views/Settings.vue b/frontend/src/views/Settings.vue index 45fcfff..afff155 100644 --- a/frontend/src/views/Settings.vue +++ b/frontend/src/views/Settings.vue @@ -163,38 +163,34 @@
{{ t('sources.shareTitle') }} - {{ t('sources.off') }} - -
+ {{ t('sources.shareHub') }} + {{ t('sources.off') }} + + {{ deploys.sync.ok ? t('sources.connected') : t('sources.failed') }} + {{ t('sources.shareWait') }}

{{ t('sources.shareWhy') }}

- - - - - -
- - +
+ {{ t('sources.shareHubUrl') }}：{{ deploys.hub_url || '—' }} + （{{ t('sources.shareHubHint') }}）
-
{{ t('sources.shareHint') }}
-
+
{{ deploys.sync.error }}
+
{{ t('sources.communitySize') }}：{{ t('sources.communityDetail', { ip: (deploys.sync.community_ips || 0).toLocaleString(), dom: (deploys.sync.community_domains || 0).toLocaleString() }) }} （{{ t('settings.last') }} {{ ago(deploys.sync.ts) }}）
-
- {{ t('sources.deploys') }}：{{ t('sources.deployed', { n: deploys.total }) }} · - {{ t('sources.online', { n: deploys.online }) }} · - {{ t('sources.offline', { n: deploys.offline }) }} +
+ {{ t('sources.deploys') }}：{{ t('sources.deployed', { n: deploys.sync.total }) }} · + {{ t('sources.online', { n: deploys.sync.online }) }} · + {{ t('sources.offline', { n: deploys.sync.offline }) }} （{{ t('sources.deployHint') }}）
+
+ {{ t('sources.shareNets') }}：{{ deploys.sync.nets }} +
{{ t('sources.shareNetsWarn') }}
+
{{ srcErr }} @@ -330,7 +326,7 @@ onMounted(async () => { }) // ---------- 外部数据源 ---------- -const src = reactive({ singbox_url: '', singbox_secret: '', proxy_ips: '', mosdns_url: '', adguard_url: '', adguard_username: '', adguard_password: '', share_url: '', share_enabled: true, share_user: '', share_password: '' }) +const src = reactive({ singbox_url: '', singbox_secret: '', proxy_ips: '', mosdns_url: '', adguard_url: '', adguard_username: '', adguard_password: '', share_enabled: true }) const deploys = ref(null) const srcErr = ref('') const srcSaved = ref(false) @@ -339,7 +335,7 @@ function fillSources() { const i = s.value.integrations Object.assign(src, { singbox_url: i.singbox_url, singbox_secret: '', proxy_ips: i.proxy_ips, mosdns_url: i.mosdns_url, adguard_url: i.adguard_url, adguard_username: i.adguard_username, adguard_password: '', - share_url: i.webdav_url, share_user: i.webdav_user || '', share_enabled: !!s.value.share_enabled, share_password: '' }) + share_enabled: !!s.value.share_enabled }) } async function saveSources() { srcErr.value = '' @@ -347,23 +343,20 @@ async function saveSources() { // 密钥留空表示不修改；要清除用“清除密钥/密码”按钮 const body = { singbox_url: src.singbox_url, proxy_ips: src.proxy_ips, mosdns_url: src.mosdns_url, adguard_url: src.adguard_url, adguard_username: src.adguard_username, - share_url: src.share_url, share_user: src.share_user } + share_enabled: src.share_enabled } if (src.singbox_secret) body.singbox_secret = src.singbox_secret if (src.adguard_password) body.adguard_password = src.adguard_password - if (src.share_password) body.webdav_password = src.share_password s.value = await http.put('/settings', body) fillSources() srcSaved.value = true setTimeout(() => (srcSaved.value = false), 1500) await testSources() + setTimeout(loadDeploys, 3000) // 打开共享开关后后台立刻同步一轮，稍后刷新状态 } catch (e) { srcErr.value = e.message } } async function clearSecret() { try { s.value = await http.put('/settings', { singbox_secret: '' }); fillSources() } catch (e) { srcErr.value = e.message } } -async function clearSharePassword() { - try { s.value = await http.put('/settings', { webdav_password: '' }); fillSources() } catch (e) { srcErr.value = e.message } -} async function clearAdguardPassword() { try { s.value = await http.put('/settings', { adguard_password: '' }); fillSources() } catch (e) { srcErr.value = e.message } } @@ -469,6 +462,18 @@ const fixes = computed(() => { } else if (c.issues.includes('no_quic')) { out.push(`/ip firewall mangle add chain=forward protocol=udp dst-port=443 connection-bytes=0-4000 ${tail}`) } + if (c.issues.includes('no_ipv6')) { + // IPv6 再加一份同样的 DNS / SNI 嗅探（镜像包仍发到 ROS-DPI 的 IPv4 地址） + const dns = `action=sniff-tzsp sniff-target=${tgt} sniff-target-port=${c.port} comment=ROSDPI-DNS` + out.push(`/ipv6 firewall mangle add chain=output protocol=udp src-port=53 ${dns}`) + out.push(`/ipv6 firewall mangle add chain=forward protocol=udp src-port=53 ${dns}`) + out.push(`/ipv6 firewall mangle add chain=forward protocol=tcp dst-port=443 connection-bytes=0-4000 ${tail}`) + out.push(`/ipv6 firewall mangle add chain=forward protocol=udp dst-port=443 connection-bytes=0-4000 ${tail}`) + } + if (c.issues.includes('fasttrack')) { + // 每条连接的前 10KB 走正常路径（握手能被镜像），之后照常 FastTrack + out.push('/ip firewall filter set [find action=fasttrack-connection] connection-bytes=10000-0') + } return [...new Set(out)] }) const fixCopied = ref(false)