Merge pull request #1 from jinyiwei2012/feat/pools-and-sentinel

feat: 代理池 + Cookie 池预热 + 日志轮转 + 配额哨兵 + 求解器修复
This commit is contained in:
nyoung authored and GitHub committed 2026-10-08 09:29:07 +08:00
commit 454bb970b8
28 files changed
+4638 -68

No files matched your search

+19 -7
View File
@@ -35,10 +35,11 @@ Go 实现,编译为**单个静态二进制**,无运行时依赖;内置 **W
- **三站点聚合**:单站点每日额度耗尽自动切换到其他站点的同款模型,额度耗尽的站点临时降级 10 分钟
- **站点额度管理**:额度按访客身份按日计算,控制台站点卡片可一键查询当前额度、重置访客身份(等同新开隐私窗口、额度即刻回满);请求撞额度时也会自动轮换访客身份恢复,无需重新求解
- **代理线路**:主线路 + 多条备用线路,支持 `http://` `https://` `socks5://`;slow-start 检测 + 失败降级 + 流中失败不重放
- **代理池**:Xray 核心(分享链接 / 订阅 / `ip:port` 端点;核心可自动下载)+ 粘性选路 + 后台健康探测;每个站点可绑定独立出口,Turnstile 求解与请求共用同一出口
- **Turnstile 自动突破**:三种 Token 获取方式 — 求解服务 API(`provider="api"`)、本地浏览器自动求解(`provider="browser"`,CDP 直连 Chrome/Edge,无需 Playwright/Docker)、手动导入 Cookie(`provider="manual"`);结果按「代理线路 × 站点」缓存,TTL 过期自动重解
- **会话自愈**:nonce / data-config 缓存 + 配额耗尽 / 验证失败时自动刷新重试
- **Web 管理台**:仪表盘、模型与站点、API Key、代理线路、Turnstile、对话调试、实时日志、设置
- **可观测**:按模型 / 站点 / 按日统计、P50/P95 延迟、最近请求、实时日志(SSE),数据持久化到 `data/`
- **可观测**:按模型 / 站点 / 按日统计、P50/P95 延迟、最近请求、实时日志(SSE)、文件日志按日压缩轮转,数据持久化到 `data/`
- **安全**:下游 Bearer / `X-Api-Key` 鉴权、每 Key 每分钟限流、每站点并发闸门、管理台独立口令
---
@@ -134,14 +135,16 @@ dsfree2api -config config.toml -check
| 字段 | 说明 |
|------|------|
| `[server]` | API 监听 `host` / `port` / `log_level` |
| `[server]` | API 监听 `host` / `port` / `log_level` / `log_file`(留空写数据目录,按日压缩轮转;`-` = 仅终端) |
| `[security].api_keys` | 下游鉴权,空数组 = 不鉴权 |
| `[admin]` | 管理台 `enabled` / `host` / `port` / `password`(留空启动时生成随机密码) |
| `[limits]` | `max_concurrent_per_site` 每站点并发、`rate_per_minute` 每 Key 每分钟限流(0 = 不限) |
| `[proxy]` | `url` 主线路、`fallback_urls` 备用线路、`slow_start_seconds` 首事件超时 |
| `[proxypool]` | 代理池总开关、健康检查间隔 / 超时 / URL、裸 `ip:port` 默认协议、Xray 路径 / 版本 / 自动下载;`[proxypool.entries.*]` 手动节点(分享链接或端点)、`[proxypool.subscriptions.*]` 订阅自动拉取 |
| `[quota]` | 配额哨兵:`enabled` / `check_seconds` / `warn_ratio`,定期用池内会话读取站点免费额度,低额告警与重置留档 |
| `[upstream]` | 超时、配置缓存 TTL、会话自动刷新、`cross_site_failover` 跨站切换、`continue_rounds` 工具调用截断续写轮数(0 = 不续写、直接截断补全) |
| `[turnstile]` | **默认 `enabled = false` 且不内置任何求解服务**;`provider` 三选一:`api`(填 `api_url` / `api_key`)、`browser`(填 `browser_path` 指向本机 Chrome/Edge)、`manual`(只用控制台导入的 Cookie),另有 Cookie TTL、重试次数 |
| `[sites.*]` | 三个站点的 `base_url` / `ajax_url` / `sitekey` / `language` |
| `[turnstile]` | **默认 `enabled = false` 且不内置任何求解服务**;`provider` 三选一:`api`(填 `api_url` / `api_key`)、`browser`(填 `browser_path` 指向本机 Chrome/Edge)、`manual`(只用控制台导入的 Cookie),另有 Cookie TTL、重试次数、单次求解超时(`timeout_seconds`)、求解服务风格(`api_style`)与 Cookie 池(`warm_enabled` / `warm_ratio` / `warm_check_seconds`,后台预热,请求零等待) |
| `[sites.*]` | 三个站点的 `base_url` / `ajax_url` / `sitekey` / `language` / `proxies`(站点级出口绑定,按顺序粘性选路)/ `warm_pool_only`(只预热池绑定线路) |
| `[models.*]` | 模型到 `site` + `upstream_id` + `bot_id` / `post_id` 的映射 |
### Turnstile 说明
@@ -154,6 +157,7 @@ dsfree2api -config config.toml -check
| `browser` | 2 · 本地浏览器自动获取 | `browser_path` | CDP 直连你配置的 Chrome / Edge:打开站点 → 注入 Turnstile widget → 可信点击 → 轮询拿到 Token;等价于 heartmore/cloudflare-solver 的做法,但**无需 Playwright / FlareSolverr / Docker**,单二进制即可。可选 `browser_user_data_dir`(真实 profile)、`browser_timezone` / `browser_locale`(对齐代理出口) |
| `manual` | 3 · 手动导入 Cookie | — | 不自动求解,只使用控制台导入的 Cookie |
- **求解服务风格**:`api_style = "ezsolver"` 时兼容本地 EzSolver 形态的求解服务(`{sitekey,siteurl,timeout}` → `{token}`),`api_key` 可留空;环回 / 内网求解地址直连(不随 `[proxy]` 线路,避免 `127.0.0.1` 被解析到隧道另一侧);
- **手动导入 Cookie**:在走同一条出口线路的浏览器里过一次验证,把 `document.cookie` 粘到控制台 **Turnstile → 手动导入 Cookie**,在 `cookie_ttl_seconds` 内免求解(Cookie 与出口 IP / UA 绑定,换线路需重新导入);
- **浏览器模式提示**:求解期间会短暂弹出浏览器窗口(`browser_headless = true` 可关闭,但过验证率更低);带账号密码的代理暂不支持 Chrome,请用 IP 白名单代理;浏览器被系统策略锁定时会报 `browser launch failed` 并附 stderr。
@@ -265,8 +269,8 @@ curl http://127.0.0.1:8000/v1/messages \
| 仪表盘 | 请求数 / 错误 / Token / P50·P95、模型用量、站点状态(含 Cookie TTL 进度条)、最近请求 |
| 模型与站点 | 编辑标签、路径、bot_id/post_id,启停模型与站点,修改站点 URL / 语言 |
| API Key | 生成 / 删除 / 复制下游 key,立即写回配置 |
| 代理线路 | 主线路、备用线路增删、连通性测试、slow-start / 并发 / 限流 / 跨站切换 |
| Turnstile | 三种 Token 获取方式的模式面板(求解服务 API / 本地浏览器自动获取 / 手动导入 Cookie)、各站 Cookie 状态与 TTL、强制刷新、求解历史 |
| 代理线路 | 主线路、备用线路增删、连通性测试、slow-start / 并发 / 限流 / 跨站切换、代理池(节点 / 订阅 / 每站点绑定 / Xray 核心状态) |
| Turnstile | 三种 Token 获取方式的模式面板(求解服务 API / 本地浏览器自动获取 / 手动导入 Cookie)、求解器风格(`api_style`)、各站 Cookie 状态与 TTL、Cookie 池预热状态、强制刷新、求解历史 |
| 对话调试 | 选模型 / 系统提示 / tools JSON,流式或非流式,带连接计时与取消 |
| 实时日志 | 级别过滤 + SSE 实时推送 |
| 设置 | 服务与上游参数、数据目录、危险操作 |
@@ -283,6 +287,7 @@ curl http://127.0.0.1:8000/v1/messages \
|------|------|------|
| `HOST` / `PORT` | API 监听地址 | `0.0.0.0` / `8000` |
| `LOG_LEVEL` | `DEBUG` `INFO` `WARN` `ERROR` | `INFO` |
| `LOG_FILE` | 日志文件路径;留空 = `<data_dir>/logs/dsfree2api.log`,`-` = 仅终端 | —(文件日志) |
| `API_KEYS` | 下游 key,逗号分隔 | —(不鉴权) |
| `PROXY_URL` | 主代理 | —(直连) |
| `PROXY_FALLBACK_URLS` | 备用代理,逗号分隔 | — |
@@ -291,6 +296,10 @@ curl http://127.0.0.1:8000/v1/messages \
| `TURNSTILE_API_KEY` | Turnstile 求解服务 key(仅在 `provider=api` 且开启时校验) | 配置文件 |
| `TURNSTILE_PROVIDER` | Token 获取方式:`api` / `browser` / `manual` | `api` |
| `TURNSTILE_BROWSER_PATH` | 浏览器模式的 Chrome / Edge 可执行文件路径 | — |
| `TURNSTILE_TIMEOUT_SECONDS` | 单次求解超时秒数(5–600) | `90` |
| `TURNSTILE_WARM_ENABLED` | Cookie 池后台预热开关 | `false` |
| `TURNSTILE_WARM_RATIO` | 剩余 TTL 低于该比例时自动续期(0–1) | `0.3` |
| `TURNSTILE_WARM_CHECK_SECONDS` | 预热巡检间隔秒数(5–3600) | `60` |
| `ADMIN_ENABLED` / `ADMIN_HOST` / `ADMIN_PORT` / `ADMIN_PASSWORD` | 管理台 | `true` / `127.0.0.1` / `8001` |
| `DATA_DIR` | 统计等运行时数据目录 | `./data` |
@@ -302,15 +311,18 @@ curl http://127.0.0.1:8000/v1/messages \
cmd/dsfree2api 入口:配置加载、日志、双 HTTP server、优雅退出
internal/config TOML 结构 + 默认值 + env 覆盖 + 校验 + 写回
internal/httpx TLS 指纹会话封装(bogdanfinn/tls-client, Chrome_120)
internal/proxypool 代理池:Xray 核心管理 / 订阅拉取 / 分享链接解析 / 粘性选路与健康探测
internal/quota 配额哨兵:站点余额轮询 / 低额告警 / 重置留档
internal/openai OpenAI 协议类型、prompt 拼接、tool_calls 解析
internal/anthropic Anthropic Messages 协议类型 + 与 OpenAI 消息的互转
internal/turnstile Turnstile 三种求解方式(API / 浏览器 CDP / 手动)+ 按线路/站点的 Cookie 缓存
internal/turnstile Turnstile 三种求解方式(API / 浏览器 CDP / 手动)+ 按线路/站点的 Cookie 缓存与后台预热(warmer)
internal/upstream 上游逆向核心:data-config/nonce → SSE → 事件翻译,
线路降级、慢启动、会话刷新、跨站切换
internal/api OpenAI / Anthropic 路由、鉴权、限流、流式写出
internal/admin 管理台后端 + go:embed 前端
internal/metrics 计数/分模型/分站点/按日/延迟分位数,JSON 持久化
internal/logbuf 环形日志 + 订阅广播(供 SSE 日志页)
internal/logfile 文件日志:按日轮转 + gzip 归档(`[server].log_file`)
```
---
+114 -10
View File
@@ -5,10 +5,12 @@ import (
"errors"
"flag"
"fmt"
"io"
"log/slog"
"net/http"
"os"
"os/signal"
"path/filepath"
"strings"
"syscall"
"time"
@@ -17,7 +19,10 @@ import (
"github.com/nyoungo/dsfree2api/internal/api"
"github.com/nyoungo/dsfree2api/internal/config"
"github.com/nyoungo/dsfree2api/internal/logbuf"
"github.com/nyoungo/dsfree2api/internal/logfile"
"github.com/nyoungo/dsfree2api/internal/metrics"
"github.com/nyoungo/dsfree2api/internal/proxypool"
"github.com/nyoungo/dsfree2api/internal/quota"
"github.com/nyoungo/dsfree2api/internal/turnstile"
"github.com/nyoungo/dsfree2api/internal/upstream"
)
@@ -57,27 +62,65 @@ func main() {
return
}
level := parseLevel(cfg.Server.LogLevel)
logs := logbuf.New(500)
logger := slog.New(logbuf.NewHandler(
slog.NewTextHandler(os.Stderr, &slog.HandlerOptions{Level: level}),
logs,
))
dataDir, err := cfg.DataDir()
if err != nil {
logger.Error("cannot create data dir", "error", err)
fmt.Fprintf(os.Stderr, "cannot create data dir: %v\n", err)
os.Exit(1)
}
level := parseLevel(cfg.Server.LogLevel)
logs := logbuf.New(500)
sinks, closeLog := logSink(cfg, dataDir)
defer closeLog()
logger := slog.New(logbuf.NewHandler(
slog.NewTextHandler(sinks, &slog.HandlerOptions{Level: level}),
logs,
))
met := metrics.New(dataDir)
met.Start()
defer met.Close()
ts := turnstile.New(cfg.Turnstile, cfg.Upstream.UserAgent)
up := upstream.New(cfg, ts, logger)
// Proxy pool: per-site egress endpoints (Xray share links, subscriptions,
// plain http/socks5) with sticky selection. No-op unless enabled.
pool := proxypool.New(cfg, dataDir, logger)
poolCtx, poolCancel := context.WithCancel(context.Background())
defer poolCancel()
pool.Start(poolCtx)
up := upstream.New(cfg, ts, logger, pool)
// Quota watchdog: every site-side quota exhaustion is logged together with
// the tokens spent on that site so far — including ones swallowed by
// cross-site failover, so the cutoff is recorded rather than hidden.
up.SetQuotaObserver(func(site, model, route string, err error) {
met.IncSiteQuota(site)
t := met.SiteTotals(site)
logger.Warn("site quota exhausted",
"site", site, "model", model, "route", route, "error", err.Error(),
"site_requests", t.Requests,
"site_prompt_tokens", t.PromptTokens,
"site_completion_tokens", t.CompletionTokens,
"site_quota_events", t.Quota)
})
// Cookie pool: keep one verified session per site × route ahead of time,
// so API requests never wait for a solve. No-op unless warm_enabled.
warmCtx, warmCancel := context.WithCancel(context.Background())
defer warmCancel()
ts.StartWarmer(warmCtx, func() []turnstile.WarmTarget { return warmTargets(cfg, up, pool) })
// Quota sentinel: poll the sites' guest-token balances for the console
// and the "quota low" alarms. No-op unless [quota].enabled.
quotaCtx, quotaCancel := context.WithCancel(context.Background())
defer quotaCancel()
quotaWatch := quota.New(cfg, logger, ts, pool)
quotaWatch.Start(quotaCtx)
apiSrv := api.New(cfg, up, met, logs, ts, logger)
adminSrv := admin.New(cfg, up, met, logs, ts, logger)
adminSrv := admin.New(cfg, up, met, logs, ts, logger, pool, quotaWatch)
mainAddr := fmt.Sprintf("%s:%d", cfg.Server.Host, cfg.Server.Port)
mainHTTP := &http.Server{
@@ -133,6 +176,67 @@ func main() {
logger.Info("bye")
}
// logSink opens the daily file log (default <data_dir>/logs/dsfree2api.log)
// and returns the combined writer plus its close function. `log_file = "-"`
// disables file logging; a setup failure degrades to stderr only.
func logSink(cfg *config.Config, dataDir string) (io.Writer, func()) {
path := strings.TrimSpace(cfg.Server.LogFile)
if path == "-" {
return os.Stderr, func() {}
}
if path == "" {
path = filepath.Join(dataDir, "logs", "dsfree2api.log")
}
w, err := logfile.New(path)
if err != nil {
fmt.Fprintf(os.Stderr, "log file disabled: %v\n", err)
return os.Stderr, func() {}
}
return io.MultiWriter(os.Stderr, w), func() { _ = w.Close() }
}
// warmTargets builds the site × route matrix the cookie pool keeps fresh:
// the sticky proxy-pool pick first, then the global primary/fallback lines.
// A site with warm_pool_only skips the global lanes and warms only its bound
// pool routes (request-time failover to the global lanes is unaffected).
func warmTargets(cfg *config.Config, up *upstream.Client, pool *proxypool.Manager) []turnstile.WarmTarget {
routes := up.Routes()
cfg.RLock()
sites := make([]config.Site, 0, len(cfg.Sites))
for _, st := range cfg.Sites {
if st != nil && st.Enabled {
sites = append(sites, *st)
}
}
cfg.RUnlock()
out := make([]turnstile.WarmTarget, 0, len(routes)*len(sites))
seen := map[string]bool{}
add := func(site config.Site, proxy string) {
key := site.Code + "|" + proxy
if seen[key] {
return
}
seen[key] = true
out = append(out, turnstile.WarmTarget{Site: site, Proxy: proxy})
}
for _, site := range sites {
if pool != nil {
if cands := pool.Candidates(site.Code); len(cands) > 0 {
add(site, cands[0].Proxy)
}
}
// warm_pool_only sites keep their cookie pool on the bound pool
// routes; the global lanes stay unwarmed.
if site.WarmPoolOnly {
continue
}
for _, r := range routes {
add(site, r.Proxy)
}
}
return out
}
func countModels(cfg *config.Config) int {
cfg.RLock()
defer cfg.RUnlock()
+61 -3
View File
@@ -1,13 +1,18 @@
# dsfree2api 示例配置。
# 复制为 config.toml 后按需修改;运行 `dsfree2api -check -config config.toml` 可校验。
# 环境变量优先级高于本文件:HOST PORT LOG_LEVEL API_KEYS PROXY_URL PROXY_FALLBACK_URLS
# 环境变量优先级高于本文件:HOST PORT LOG_LEVEL LOG_FILE API_KEYS PROXY_URL PROXY_FALLBACK_URLS
# PROXY_SLOW_START_SECONDS TURNSTILE_ENABLED TURNSTILE_API_KEY TURNSTILE_PROVIDER
# TURNSTILE_BROWSER_PATH ADMIN_ENABLED ADMIN_HOST ADMIN_PORT ADMIN_PASSWORD DATA_DIR
# TURNSTILE_BROWSER_PATH TURNSTILE_TIMEOUT_SECONDS TURNSTILE_WARM_ENABLED
# TURNSTILE_WARM_RATIO TURNSTILE_WARM_CHECK_SECONDS
# ADMIN_ENABLED ADMIN_HOST ADMIN_PORT ADMIN_PASSWORD DATA_DIR
[server]
host = "0.0.0.0"
port = 8000
log_level = "INFO"
# 日志文件:留空写入 <data_dir>/logs/dsfree2api.log;填路径则写到该文件;
# "-" = 仅终端。日志按天轮转,旧文件压缩为 <名称>-YYYY-MM-DD.log.gz。
log_file = ""
[security]
# 下游鉴权。空数组 = 不鉴权;建议部署时换成自己的 key。
@@ -33,6 +38,37 @@ fallback_urls = []
# 主线路允许的首事件等待秒数(>0 且存在备用线路时生效)。
slow_start_seconds = 8
[proxypool]
# 代理池:为每个站点绑定独立出口(Xray 分享链接 / 订阅 / ip:port 端点)。
# 选路为粘性:优先使用绑定列表中第一个健康节点,失败才切换;未绑定或
# 全部不健康时回退到上面的 [proxy] 主/备线路。Turnstile 求解与请求共用
# 同一出口(Cookie 与出口 IP 绑定,换出口后会自动重新求解)。
enabled = false
check_interval_seconds = 120 # 健康检查间隔
check_timeout_seconds = 10 # 单次探测超时
check_url = "https://www.gstatic.com/generate_204"
default_scheme = "http" # 裸 ip:port 的默认协议(http / socks5)
# Xray 核心:留空 xray_path 且开启自动下载时,首次使用会下载到数据目录
# (data/xray.exe;核心配置写到 data/config.json)。分享链接(vless / vmess
# / trojan / ss)由它承载;纯 ip:port / socks5:// 端点无需核心。
xray_path = ""
xray_version = "" # 留空 = 最新版
xray_auto_download = true
# 手动节点(名称限字母 / 数字 / - / _):
# [proxypool.entries.node-a]
# link = "vless://uuid@host:443?type=ws&security=tls&sni=example.com&path=/ws#node-a"
# link = "socks5://user:pass@1.2.3.4:1080"
# link = "1.2.3.4:8080" # 使用 default_scheme
# link = "1.2.3.4:8080:user:pass"
# enabled = true # 缺省即启用
# 订阅(按 interval_minutes 自动刷新;节点名形如 <订阅名>#xxxxxxxx):
# [proxypool.subscriptions.home]
# url = "https://example.com/api/v1/client/subscribe?token=…"
# interval_minutes = 60
# enabled = true
[upstream]
timeout = 60.0
stream_timeout = 300.0
@@ -56,7 +92,8 @@ continue_rounds = 20
# manual —— 只用控制台手动导入的 Cookie(方式 3)
enabled = false
provider = "api"
# 方式 1:求解服务 API
# 方式 1:求解服务 API(api_style = "sync" 为默认 CapSolver 形状;
# api_style = "ezsolver" 适配本地 EzSolver 服务,api_key 可留空)
# api_url = "https://your-solver.example.com/turnstile/sync"
# api_key = "REPLACE_ME"
# 方式 2:本地浏览器自动获取(CDP 直连,无需 Playwright / Docker)
@@ -67,10 +104,26 @@ provider = "api"
# browser_locale = "" # 可选:如 "ja-JP",留空取站点语言
sitekey = "0x4AAAAAADlLZ3ljqZP6cQwq"
action = "chat"
# 求解超时:单次求解(浏览器 / 求解服务 API)的最长等待秒数,5–600。
timeout_seconds = 90
cookie_ttl_seconds = 10800
retries = 5
retry_backoff_seconds = 1.5
# ── Cookie 池(后台预热)─────────────────────────────────
# 开启后,后台按 warm_check_seconds 巡检:Cookie 缺失 / 过期 /
# 剩余 TTL 低于 warm_ratio 时主动重解并写回缓存,请求路径不再等待求解。
# provider = "manual" 时不预热(手动导入的 Cookie 无法自动续期)。
warm_enabled = false
warm_ratio = 0.3 # 剩余 TTL 低于该比例(0.05–0.95)时触发预热
warm_check_seconds = 60 # 后台巡检间隔秒数(5–3600)
# ── 配额哨兵(站点免费额度实时监控)──────────────
# 用与聊天相同的出口 Cookie 定期查询站点余额接口(每日免费额度),结果
# 写入日志与后台 /api/overview 的 quota 区域;剩余低于 warn_ratio 时告警。
[quota]
enabled = false
check_seconds = 300
warn_ratio = 0.2
[runtime]
data_dir = "./data"
@@ -80,6 +133,11 @@ base_url = "https://deepseek.de"
ajax_url = "https://deepseek.de/wp-admin/admin-ajax.php"
sitekey = "0x4AAAAAADlLZ3ljqZP6cQwq"
language = "de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7"
# 站点级出口绑定(代理池条目名或 "sub:订阅名",按顺序粘性选路);
# 留空 = 使用 [proxy] 主/备线路。三个站点可各自独立配置。
# proxies = ["node-a", "sub:home"]
# 只预热代理池线路(不预热 [proxy] 主/备;请求时仍可回落):
# warm_pool_only = true
[sites.es]
base_url = "https://deepseek.es"
+285
View File
@@ -9,12 +9,14 @@ import (
"fmt"
"io"
"net/http"
"regexp"
"strings"
"time"
"github.com/nyoungo/dsfree2api/internal/config"
"github.com/nyoungo/dsfree2api/internal/httpx"
"github.com/nyoungo/dsfree2api/internal/openai"
"github.com/nyoungo/dsfree2api/internal/proxypool"
"github.com/nyoungo/dsfree2api/internal/upstream"
)
@@ -233,12 +235,20 @@ func (s *Server) handleActions(w http.ResponseWriter, r *http.Request) {
Challenge string `json:"challenge_action"`
Legacy string `json:"ts_action"`
CookieTT int `json:"cookie_ttl_seconds"`
APIStyle *string `json:"api_style"`
BrowserPath *string `json:"browser_path"`
BrowserHeadless *bool `json:"browser_headless"`
BrowserUserDataDir *string `json:"browser_user_data_dir"`
BrowserTimezone *string `json:"browser_timezone"`
BrowserLocale *string `json:"browser_locale"`
TimeoutSeconds *int `json:"timeout_seconds"`
Retries *int `json:"retries"`
RetryBackoffSeconds *float64 `json:"retry_backoff_seconds"`
WarmEnabled *bool `json:"warm_enabled"`
WarmRatio *float64 `json:"warm_ratio"`
WarmCheckSeconds *int `json:"warm_check_seconds"`
}
_ = json.Unmarshal(body, &p)
// Build the candidate config first: a rejected save must not leave
@@ -275,9 +285,30 @@ func (s *Server) handleActions(w http.ResponseWriter, r *http.Request) {
if act := firstNonEmpty(p.Challenge, p.Legacy); act != "" {
tc.Action = act
}
if p.APIStyle != nil {
tc.APIStyle = strings.ToLower(strings.TrimSpace(*p.APIStyle))
}
if p.CookieTT > 0 {
tc.CookieTTLSeconds = p.CookieTT
}
if p.TimeoutSeconds != nil {
tc.TimeoutSeconds = clampInt(*p.TimeoutSeconds, 5, 600)
}
if p.Retries != nil {
tc.Retries = clampInt(*p.Retries, 1, 20)
}
if p.RetryBackoffSeconds != nil {
tc.RetryBackoffSeconds = clampFloat(*p.RetryBackoffSeconds, 0, 60)
}
if p.WarmEnabled != nil {
tc.WarmEnabled = *p.WarmEnabled
}
if p.WarmRatio != nil {
tc.WarmRatio = clampFloat(*p.WarmRatio, 0.05, 0.95)
}
if p.WarmCheckSeconds != nil {
tc.WarmCheckSeconds = clampInt(*p.WarmCheckSeconds, 5, 3600)
}
tc.Enabled = p.Enabled
if tc.Enabled {
// provider-specific requirements mirror config.Validate()
@@ -396,6 +427,218 @@ func (s *Server) handleActions(w http.ResponseWriter, r *http.Request) {
_ = json.Unmarshal(body, &p)
writeJSON(w, http.StatusOK, s.rotateGuest(p.Site))
case "pool_set_config":
var p struct {
Enabled *bool `json:"enabled"`
CheckIntervalSeconds *int `json:"check_interval_seconds"`
CheckTimeoutSeconds *int `json:"check_timeout_seconds"`
CheckURL *string `json:"check_url"`
DefaultScheme *string `json:"default_scheme"`
XrayPath *string `json:"xray_path"`
XrayVersion *string `json:"xray_version"`
XrayAutoDownload *bool `json:"xray_auto_download"`
}
_ = json.Unmarshal(body, &p)
s.cfg.Lock()
pc := s.cfg.ProxyPool
if p.Enabled != nil {
pc.Enabled = *p.Enabled
}
if p.CheckIntervalSeconds != nil {
pc.CheckIntervalSeconds = clampInt(*p.CheckIntervalSeconds, 10, 3600)
}
if p.CheckTimeoutSeconds != nil {
pc.CheckTimeoutSeconds = clampInt(*p.CheckTimeoutSeconds, 2, 120)
}
if p.CheckURL != nil {
pc.CheckURL = strings.TrimSpace(*p.CheckURL)
}
if p.DefaultScheme != nil {
pc.DefaultScheme = strings.ToLower(strings.TrimSpace(*p.DefaultScheme))
}
if p.XrayPath != nil {
pc.XrayPath = strings.TrimSpace(*p.XrayPath)
}
if p.XrayVersion != nil {
pc.XrayVersion = strings.TrimSpace(*p.XrayVersion)
}
if p.XrayAutoDownload != nil {
pc.XrayAutoDownload = *p.XrayAutoDownload
}
s.cfg.ProxyPool = pc
s.cfg.Unlock()
s.poolReload()
s.save(w)
case "pool_add_entry":
var p struct {
Name string `json:"name"`
Link string `json:"link"`
}
_ = json.Unmarshal(body, &p)
p.Name, p.Link = strings.TrimSpace(p.Name), strings.TrimSpace(p.Link)
if !validPoolName(p.Name) {
fail(w, "name must match [A-Za-z0-9_-]{1,64}")
return
}
if p.Link == "" {
fail(w, "link is required")
return
}
s.cfg.RLock()
scheme := s.cfg.ProxyPool.DefaultScheme
_, exists := s.cfg.ProxyPool.Entries[p.Name]
s.cfg.RUnlock()
if exists {
fail(w, "entry already exists: "+p.Name)
return
}
if _, err := proxypool.ParseNode(p.Link, scheme); err != nil {
fail(w, "node parse: "+err.Error())
return
}
s.cfg.Lock()
if s.cfg.ProxyPool.Entries == nil {
s.cfg.ProxyPool.Entries = map[string]*config.ProxyEntry{}
}
s.cfg.ProxyPool.Entries[p.Name] = &config.ProxyEntry{Link: p.Link, Enabled: true}
s.cfg.Unlock()
s.poolReload()
s.save(w)
case "pool_toggle_entry", "pool_toggle_sub":
var p struct {
Name string `json:"name"`
Enabled bool `json:"enabled"`
}
_ = json.Unmarshal(body, &p)
s.cfg.Lock()
ok := false
if req.Action == "pool_toggle_entry" {
if e := s.cfg.ProxyPool.Entries[p.Name]; e != nil {
e.Enabled = p.Enabled
ok = true
}
} else {
if sc := s.cfg.ProxyPool.Subscriptions[p.Name]; sc != nil {
sc.Enabled = p.Enabled
ok = true
}
}
s.cfg.Unlock()
if !ok {
fail(w, "not found: "+p.Name)
return
}
s.poolReload()
s.save(w)
case "pool_delete_entry", "pool_delete_sub":
var p struct {
Name string `json:"name"`
}
_ = json.Unmarshal(body, &p)
ref := p.Name
if req.Action == "pool_delete_sub" {
ref = "sub:" + p.Name
}
s.cfg.Lock()
if req.Action == "pool_delete_entry" {
delete(s.cfg.ProxyPool.Entries, p.Name)
} else {
delete(s.cfg.ProxyPool.Subscriptions, p.Name)
}
for _, st := range s.cfg.Sites {
if st != nil {
st.Proxies = removeString(st.Proxies, ref)
}
}
s.cfg.Unlock()
s.poolReload()
s.save(w)
case "pool_add_sub":
var p struct {
Name string `json:"name"`
URL string `json:"url"`
IntervalMinutes int `json:"interval_minutes"`
}
_ = json.Unmarshal(body, &p)
p.Name, p.URL = strings.TrimSpace(p.Name), strings.TrimSpace(p.URL)
if !validPoolName(p.Name) {
fail(w, "name must match [A-Za-z0-9_-]{1,64}")
return
}
if p.URL == "" {
fail(w, "url is required")
return
}
s.cfg.Lock()
if s.cfg.ProxyPool.Subscriptions == nil {
s.cfg.ProxyPool.Subscriptions = map[string]*config.ProxySubscription{}
}
if _, exists := s.cfg.ProxyPool.Subscriptions[p.Name]; exists {
s.cfg.Unlock()
fail(w, "subscription already exists: "+p.Name)
return
}
if p.IntervalMinutes <= 0 {
p.IntervalMinutes = 60
}
s.cfg.ProxyPool.Subscriptions[p.Name] = &config.ProxySubscription{
URL: p.URL, Enabled: true,
IntervalMinutes: clampInt(p.IntervalMinutes, 5, 1440),
}
s.cfg.Unlock()
s.poolReload()
s.save(w)
case "pool_set_site":
var p struct {
Site string `json:"site"`
Proxies []string `json:"proxies"`
}
_ = json.Unmarshal(body, &p)
clean := make([]string, 0, len(p.Proxies))
for _, item := range p.Proxies {
if item = strings.TrimSpace(item); item != "" {
clean = append(clean, item)
}
}
s.cfg.Lock()
st, ok := s.cfg.Sites[p.Site]
if ok && st != nil {
for _, item := range clean {
if name, isSub := strings.CutPrefix(item, "sub:"); isSub {
if s.cfg.ProxyPool.Subscriptions[name] == nil {
s.cfg.Unlock()
fail(w, "unknown subscription: "+name)
return
}
} else if s.cfg.ProxyPool.Entries[item] == nil {
s.cfg.Unlock()
fail(w, "unknown entry: "+item)
return
}
}
st.Proxies = clean
}
s.cfg.Unlock()
if !ok || st == nil {
fail(w, "unknown site: "+p.Site)
return
}
s.poolReload()
s.save(w)
case "pool_refresh":
if s.pool == nil {
fail(w, "proxy pool is not available")
return
}
s.pool.RefreshNow(context.Background())
writeJSON(w, http.StatusOK, map[string]any{"ok": true})
default:
fail(w, "unknown action: "+req.Action)
}
@@ -673,3 +916,45 @@ func randomToken(n int) string {
}
return hex.EncodeToString(b)
}
func clampInt(v, lo, hi int) int {
if v < lo {
return lo
}
if v > hi {
return hi
}
return v
}
func clampFloat(v, lo, hi float64) float64 {
if v < lo {
return lo
}
if v > hi {
return hi
}
return v
}
var poolNameRe = regexp.MustCompile(`^[A-Za-z0-9_-]{1,64}$`)
func validPoolName(name string) bool { return poolNameRe.MatchString(name) }
// removeString drops every occurrence of want from list (in place copy).
func removeString(list []string, want string) []string {
out := list[:0:0]
for _, v := range list {
if v != want {
out = append(out, v)
}
}
return out
}
// poolReload pushes pool-relevant config changes into the live manager.
func (s *Server) poolReload() {
if s.pool != nil {
s.pool.Reload()
}
}
+22 -4
View File
@@ -15,6 +15,8 @@ import (
"github.com/nyoungo/dsfree2api/internal/config"
"github.com/nyoungo/dsfree2api/internal/logbuf"
"github.com/nyoungo/dsfree2api/internal/metrics"
"github.com/nyoungo/dsfree2api/internal/proxypool"
"github.com/nyoungo/dsfree2api/internal/quota"
"github.com/nyoungo/dsfree2api/internal/turnstile"
"github.com/nyoungo/dsfree2api/internal/upstream"
)
@@ -31,6 +33,8 @@ type Server struct {
met *metrics.Recorder
logs *logbuf.Buffer
ts *turnstile.Solver
pool *proxypool.Manager
quota *quota.Watcher
log *slog.Logger
start time.Time
@@ -39,7 +43,7 @@ type Server struct {
password string
}
func New(cfg *config.Config, up *upstream.Client, met *metrics.Recorder, logs *logbuf.Buffer, ts *turnstile.Solver, log *slog.Logger) *Server {
func New(cfg *config.Config, up *upstream.Client, met *metrics.Recorder, logs *logbuf.Buffer, ts *turnstile.Solver, log *slog.Logger, pool *proxypool.Manager, qw *quota.Watcher) *Server {
if log == nil {
log = slog.Default()
}
@@ -50,7 +54,7 @@ func New(cfg *config.Config, up *upstream.Client, met *metrics.Recorder, logs *l
generated = true
}
s := &Server{
cfg: cfg, up: up, met: met, logs: logs, ts: ts, log: log,
cfg: cfg, up: up, met: met, logs: logs, ts: ts, pool: pool, quota: qw, log: log,
start: time.Now(), tokens: map[string]time.Time{}, password: pw,
}
if generated {
@@ -185,7 +189,7 @@ func (s *Server) handleOverview(w http.ResponseWriter, _ *http.Request) {
sites = append(sites, map[string]any{
"code": code, "base_url": st.BaseURL, "ajax_url": st.AJAXURL,
"language": st.Language, "sitekey": st.SiteKey, "enabled": st.Enabled,
"verify_action": st.VerifyAction,
"verify_action": st.VerifyAction, "proxies": st.Proxies,
})
}
proxyCfg := struct {
@@ -201,12 +205,23 @@ func (s *Server) handleOverview(w http.ResponseWriter, _ *http.Request) {
keys := append([]string(nil), s.cfg.Security.APIKeys...)
s.cfg.RUnlock()
var poolStatus any
if s.pool != nil {
poolStatus = s.pool.Status()
}
var quotaStatus any
if s.quota != nil {
quotaStatus = s.quota.Snapshot()
}
writeJSON(w, http.StatusOK, map[string]any{
"metrics": s.met.Snapshot(),
"models": models,
"sites": sites,
"proxy": proxyCfg,
"turnstile": map[string]any{"config": turnstileCfg, "status": s.ts.Status(siteCodes), "history": s.ts.History()},
"turnstile": map[string]any{"config": turnstileCfg, "status": s.ts.Status(siteCodes), "history": s.ts.History(), "pool": s.ts.PoolStatus()},
"proxypool": poolStatus,
"quota": quotaStatus,
"keys": keys,
"cache": map[string]any{"chat_config_entries": s.up.ConfigCacheSize(), "routes": routeList(s.up)},
"system": map[string]any{
@@ -269,6 +284,9 @@ func (s *Server) handlePutConfig(w http.ResponseWriter, r *http.Request) {
// applyRuntime pushes a freshly edited config into the live subsystems.
func (s *Server) applyRuntime() {
s.ts.SetConfig(s.currentTurnstile())
if s.pool != nil {
s.pool.Reload()
}
}
func (s *Server) currentTurnstile() config.Turnstile {
+194 -2
View File
@@ -328,6 +328,7 @@ const mask = k => k.length > 16 ? k.slice(0, 7) + "…" + k.slice(-6) : k;
PAGES.proxy = async () => {
const ov = await fetchOverview();
const p = ov.proxy;
const pool = ov.proxypool || { xray: {}, entries: [], subscriptions: [], sticky: {} };
view.innerHTML = `
<div class="grid g2">
<div class="card"><h3>主线路</h3>
@@ -355,6 +356,49 @@ PAGES.proxy = async () => {
<div class="row"><input id="px-fb" placeholder="socks5://..." style="width:280px"><button class="btn" id="px-add">添加</button></div></div>
<div class="table-wrap mt"><table><thead><tr><th>#</th><th>地址</th><th></th></tr></thead><tbody id="tb-fb"></tbody></table></div>
<div class="dim mt">当前生效顺序:${ov.cache.routes.map(r => `<span class="tag">${esc(r.name)} → ${esc(r.proxy)}</span>`).join(" ")}</div>
</div>
<div class="grid g2 mt">
<div class="card"><h3>代理池 · Xray 核心</h3>
<div class="row">
<div class="field"><label>池开关</label>${switchHTML("pool-en", pool.enabled)}</div>
<div class="field"><label>检查间隔(秒)</label><input id="pool-check-int" type="number" min="10" value="${pool.check_interval_seconds || 120}"></div>
<div class="field"><label>检查超时(秒)</label><input id="pool-check-to" type="number" min="2" value="${pool.check_timeout_seconds || 10}"></div>
</div>
<div class="row">
<div class="field"><label>健康检查 URL</label><input id="pool-check-url" value="${esc(pool.check_url || "")}" placeholder="https://www.gstatic.com/generate_204"></div>
<div class="field"><label>裸 ip:port 默认协议</label><input id="pool-scheme" value="${esc(pool.default_scheme || "http")}" placeholder="http / socks5"></div>
</div>
<div class="row">
<div class="field"><label>Xray 路径(留空 = 自动)</label><input id="pool-xray-path" value="${esc(pool.xray_path || "")}" placeholder="留空自动下载到数据目录"></div>
<div class="field"><label>版本(留空 = latest)</label><input id="pool-xray-ver" value="${esc(pool.xray_version || "")}"></div>
<div class="field"><label>自动下载</label>${switchHTML("pool-xray-auto", pool.xray_auto_download)}</div>
</div>
<div class="row mt">
<button class="btn primary" id="pool-save">保存池配置</button>
<button class="btn" id="pool-refresh">刷新节点与健康检查</button>
</div>
<div class="dim mt">Xray:<span class="pill ${pool.xray.running ? "on" : "off"}">${pool.xray.running ? "运行中 · pid " + pool.xray.pid : "未运行"}</span>
${pool.xray.error ? `<span style="color:var(--err)"> ${esc(pool.xray.error)}</span>` : ""}
${pool.last_error ? `<br><span style="color:var(--err)">${esc(pool.last_error)}</span>` : ""}</div>
</div>
<div class="card"><h3>节点与端点</h3>
<div class="row"><input id="pool-name" placeholder="名称(字母数字-_)" style="width:180px">
<input id="pool-link" placeholder="vless://… 或 ip:port 或 socks5://user:pass@host:port" style="flex:1;min-width:200px">
<button class="btn" id="pool-add">添加</button></div>
<div class="table-wrap mt"><table><thead><tr><th>名称</th><th>类型</th><th>目标</th><th>健康</th><th class="num">延迟</th><th></th></tr></thead><tbody id="tb-pool"></tbody></table></div>
<div class="dim mt">分享链接(vless / vmess / trojan / ss)由内置 Xray 核心承载;ip:port / socks5:// 等端点直接使用。</div>
</div>
</div>
<div class="card mt"><div class="toolbar"><h3 style="margin:0">订阅</h3>
<div class="row"><input id="sub-name" placeholder="名称" style="width:150px">
<input id="sub-url" placeholder="https://…/sub?token=…" style="width:360px">
<input id="sub-int" type="number" min="5" value="60" title="刷新间隔(分钟)" style="width:90px">
<button class="btn" id="sub-add">添加订阅</button></div></div>
<div class="table-wrap mt"><table><thead><tr><th>名称</th><th>地址</th><th class="num">间隔</th><th class="num">节点</th><th>上次拉取</th><th></th></tr></thead><tbody id="tb-subs"></tbody></table></div>
</div>
<div class="card mt"><h3>站点绑定(按顺序,粘性选路)</h3>
<div class="table-wrap"><table><thead><tr><th>站点</th><th>绑定(逗号分隔:名称 或 sub:订阅名)</th><th>当前出口</th><th></th></tr></thead><tbody id="tb-bind"></tbody></table></div>
<div class="dim mt">未绑定、或绑定节点全部不健康时,回退到主线路 / 备用线路。Cookie 与出口 IP 绑定:切换出口后该站点会在下一轮巡检重新求解 Turnstile。</div>
</div>`;
const renderFb = list => {
@@ -369,6 +413,98 @@ PAGES.proxy = async () => {
};
renderFb(p.fallbacks || []);
const renderPool = () => {
const entries = pool.entries || [];
$("#tb-pool").innerHTML = entries.length ? entries.map(e => {
const health = e.parse_error ? `<span class="pill off" title="${esc(e.parse_error)}">解析失败</span>`
: e.checked ? (e.healthy ? '<span class="pill on">健康</span>' : `<span class="pill off" title="${esc(e.last_error || "")}">不可用</span>`)
: '<span class="pill">未探测</span>';
return `<tr><td><code>${esc(e.name)}</code>${e.source ? ` <span class="dim">← ${esc(e.source)}</span>` : ""}</td>
<td>${esc(e.kind === "url" ? e.scheme : (e.scheme || "xray"))}</td>
<td class="dim">${esc(e.target || "")}</td>
<td>${health}</td><td class="num">${e.latency_ms ? e.latency_ms + "ms" : "–"}</td>
<td class="right"><button class="btn small" data-ptoggle="${esc(e.name)}" data-pen="${e.enabled ? "0" : "1"}">${e.enabled ? "停用" : "启用"}</button>
${e.source ? "" : `<button class="btn small danger" data-pdel="${esc(e.name)}">删除</button>`}</td></tr>`;
}).join("") : '<tr><td colspan="6" class="empty">尚无节点 —— 添加分享链接或 ip:port 端点</td></tr>';
$$("[data-ptoggle]").forEach(b => b.onclick = async () => {
try { await api("/api/actions", { body: { action: "pool_toggle_entry", name: b.dataset.ptoggle, enabled: b.dataset.pen === "1" } }); PAGES.proxy(); }
catch (e) { toast(e.message, "err"); }
});
$$("[data-pdel]").forEach(b => b.onclick = async () => {
if (!confirm("删除节点 " + b.dataset.pdel + "?")) return;
try { await api("/api/actions", { body: { action: "pool_delete_entry", name: b.dataset.pdel } }); PAGES.proxy(); }
catch (e) { toast(e.message, "err"); }
});
const subs = pool.subscriptions || [];
$("#tb-subs").innerHTML = subs.length ? subs.map(s => `
<tr><td><code>${esc(s.name)}</code></td><td class="dim">${esc(s.url)}</td>
<td class="num">${s.interval_minutes || 60}m</td><td class="num">${s.nodes || 0}</td>
<td class="dim">${s.fetched_at ? timeStr(s.fetched_at * 1000) : "未拉取"}${s.last_error ? ` <span style="color:var(--err)">${esc(String(s.last_error).slice(0, 60))}</span>` : ""}</td>
<td class="right"><button class="btn small" data-stoggle="${esc(s.name)}" data-sen="${s.enabled ? "0" : "1"}">${s.enabled ? "停用" : "启用"}</button>
<button class="btn small danger" data-sdel="${esc(s.name)}">删除</button></td></tr>`).join("")
: '<tr><td colspan="6" class="empty">尚无订阅</td></tr>';
$$("[data-stoggle]").forEach(b => b.onclick = async () => {
try { await api("/api/actions", { body: { action: "pool_toggle_sub", name: b.dataset.stoggle, enabled: b.dataset.sen === "1" } }); PAGES.proxy(); }
catch (e) { toast(e.message, "err"); }
});
$$("[data-sdel]").forEach(b => b.onclick = async () => {
if (!confirm("删除订阅 " + b.dataset.sdel + "?")) return;
try { await api("/api/actions", { body: { action: "pool_delete_sub", name: b.dataset.sdel } }); PAGES.proxy(); }
catch (e) { toast(e.message, "err"); }
});
const sticky = pool.sticky || {};
$("#tb-bind").innerHTML = (ov.sites || []).map(s => {
const cur = sticky[s.code];
const curEntry = entries.find(e => e.name === cur);
return `<tr><td><b>${esc(s.code.toUpperCase())}</b></td>
<td><input data-bind="${esc(s.code)}" value="${esc((s.proxies || []).join(", "))}" placeholder="留空 = 全局线路"></td>
<td class="dim">${cur ? `${esc(cur)} → ${esc(curEntry ? curEntry.target : "?")}` : "全局线路"}</td>
<td class="right"><button class="btn small" data-bsave="${esc(s.code)}">保存</button></td></tr>`;
}).join("");
$$("[data-bsave]").forEach(b => b.onclick = async () => {
const input = $(`[data-bind="${CSS.escape(b.dataset.bsave)}"]`);
const list = input.value.split(",").map(x => x.trim()).filter(Boolean);
try { await api("/api/actions", { body: { action: "pool_set_site", site: b.dataset.bsave, proxies: list } }); toast("已保存", "ok"); PAGES.proxy(); }
catch (e) { toast(e.message, "err"); }
});
};
renderPool();
$("#pool-save").onclick = async () => {
try {
await api("/api/actions", { body: {
action: "pool_set_config",
enabled: $("#pool-en").checked,
check_interval_seconds: +$("#pool-check-int").value || 120,
check_timeout_seconds: +$("#pool-check-to").value || 10,
check_url: $("#pool-check-url").value.trim(),
default_scheme: $("#pool-scheme").value.trim() || "http",
xray_path: $("#pool-xray-path").value.trim(),
xray_version: $("#pool-xray-ver").value.trim(),
xray_auto_download: $("#pool-xray-auto").checked } });
toast("池配置已保存", "ok"); PAGES.proxy();
} catch (e) { toast(e.message, "err"); }
};
$("#pool-refresh").onclick = async () => {
const b = $("#pool-refresh"); b.disabled = true; b.textContent = "刷新中…";
try { await api("/api/actions", { body: { action: "pool_refresh" } }); toast("已在后台拉取订阅并检查健康,稍后重进本页查看", "ok"); PAGES.proxy(); }
catch (e) { toast(e.message, "err"); b.disabled = false; b.textContent = "刷新节点与健康检查"; }
};
$("#pool-add").onclick = async () => {
try {
await api("/api/actions", { body: { action: "pool_add_entry", name: $("#pool-name").value.trim(), link: $("#pool-link").value.trim() } });
toast("已添加", "ok"); PAGES.proxy();
} catch (e) { toast(e.message, "err"); }
};
$("#sub-add").onclick = async () => {
try {
await api("/api/actions", { body: { action: "pool_add_sub", name: $("#sub-name").value.trim(), url: $("#sub-url").value.trim(), interval_minutes: +$("#sub-int").value || 60 } });
toast("已添加", "ok"); PAGES.proxy();
} catch (e) { toast(e.message, "err"); }
};
$("#px-save").onclick = async () => {
try {
await api("/api/config", { method: "PUT", body: await merged(c => { c.proxy.url = $("#px-primary").value.trim(); }) });
@@ -406,6 +542,7 @@ async function merged(mutate) {
}
PAGES.turnstile = async () => {
if (overviewTimer) { clearInterval(overviewTimer); overviewTimer = null; }
const ov = await fetchOverview();
const t = ov.turnstile.config;
view.innerHTML = `
@@ -426,6 +563,10 @@ PAGES.turnstile = async () => {
<div class="row">
<div class="field"><label>Sitekey</label><input id="ts-sitekey" value="${esc(t.sitekey)}"></div>
<div class="field"><label>Action</label><input id="ts-action" value="${esc(t.action)}"></div>
<div class="field"><label>API 风格</label><select id="ts-style">
<option value="sync">sync(CapSolver 形状)</option>
<option value="ezsolver">ezsolver(本地 EzSolver)</option>
</select></div>
</div>
</div>
<div id="ts-browser" class="ts-panel" style="display:none">
@@ -459,6 +600,20 @@ PAGES.turnstile = async () => {
<div class="dim">Cookie 与出口 IP / UA 绑定:请在同一代理线路的浏览器中获取。导入后在 TTL 内免求解,状态见右侧「已验证 Cookie」。</div>
<button class="btn primary mt" id="im-save">导入</button>
</div>
<div class="ts-panel">
<div class="ts-panel-title">求解与 Cookie 池</div>
<div class="row">
<div class="field"><label>求解超时(秒)</label><input id="ts-timeout" type="number" min="5" max="600" value="${t.timeout_seconds || 90}"></div>
<div class="field"><label>失败重试(次)</label><input id="ts-retries" type="number" min="1" max="20" value="${t.retries || 5}"></div>
<div class="field"><label>重试退避(秒)</label><input id="ts-backoff" type="number" min="0" max="60" step="0.5" value="${t.retry_backoff_seconds == null ? 1.5 : t.retry_backoff_seconds}"></div>
</div>
<div class="row">
<div class="field"><label>Cookie 池(后台预热)</label>${switchHTML("ts-warm", !!t.warm_enabled)}</div>
<div class="field"><label>预热阈值(剩余 %)</label><input id="ts-ratio" type="number" min="5" max="95" value="${Math.round((t.warm_ratio == null ? 0.3 : t.warm_ratio) * 100)}"></div>
<div class="field"><label>检查间隔(秒)</label><input id="ts-check" type="number" min="5" max="3600" value="${t.warm_check_seconds || 60}"></div>
</div>
<div class="dim">Cookie 池开启后:后台每隔检查间隔巡检一次,剩余 TTL 低于阈值时主动重解,请求路径不再等待求解耗时(manual 方式不预热)。</div>
</div>
<div class="dim" id="ts-hint" style="margin-top:10px"></div>
<button class="btn primary mt" id="ts-save">保存配置</button>
</div>
@@ -468,7 +623,11 @@ PAGES.turnstile = async () => {
<button class="btn" id="ts-refresh">强制刷新全部</button>
<button class="btn danger" id="ts-clear">清空 Cookie</button>
</div>
<div class="dim mt">缓存 TTL ${t.cookie_ttl_seconds}s · 求解失败重试 ${t.retries} 次 · 退避 ${t.retry_backoff_seconds}s</div>
<div class="dim mt">缓存 TTL ${t.cookie_ttl_seconds}s · 求解超时 ${t.timeout_seconds || 90}s · 重试 ${t.retries} 次 · 退避 ${t.retry_backoff_seconds}s</div>
</div>
<div class="card"><h3>Cookie 池</h3>
<div class="dim" id="ts-pool-state"></div>
<div id="ts-pool" class="grid mt" style="gap:8px"></div>
</div>
</div>
<div class="card mt"><h3>求解历史</h3><div class="table-wrap"><table>
@@ -489,13 +648,39 @@ PAGES.turnstile = async () => {
<td><span class="pill ${h.ok ? "on" : "off"}">${h.ok ? "成功" : "失败"}</span>${h.error ? ` <span class="dim">${esc(h.error.slice(0, 70))}</span>` : ""}</td></tr>`).join("")
: '<tr><td colspan="5" class="empty">尚无记录</td></tr>';
const renderPool = (o) => {
const pool = (o.turnstile && o.turnstile.pool) || { entries: [] };
const label = $("#ts-pool-state");
if (label) {
label.textContent = `状态:${pool.enabled ? (pool.running ? "运行中" : "已启用,等待下一轮巡检") : "未开启"} · 阈值 剩余 ${Math.round((pool.ratio || 0.3) * 100)}% · 每 ${pool.check_seconds || 60}s 巡检一次`;
}
const box = $("#ts-pool");
if (!box) return;
const entries = pool.entries || [];
box.innerHTML = entries.length ? entries.map(e => {
const state = e.warming ? '<span class="pill warn">预热中</span>'
: e.valid ? '<span class="pill on">就绪</span>'
: e.last_error ? '<span class="pill off">求解失败</span>' : '<span class="pill off">未建立</span>';
const left = e.valid ? `剩余 ${Math.max(0, Math.round(e.remaining_s / 60))} 分钟` : "等待预热";
const next = e.next_warm_at ? `下次 ${timeStr(e.next_warm_at * 1000)}` : "—";
const last = e.last_at ? `上次 ${timeStr(e.last_at * 1000)} · ${fmtMs(e.last_ms)}` : "尚未求解";
return `<div class="card" style="padding:10px">
<div class="toolbar"><div><b>${esc(e.site.toUpperCase())}</b> <span class="dim">${esc(e.route)}</span></div>${state}</div>
<div class="dim mt">${left} · ${next}<br>${last}${e.last_error ? `<br><span style="color:var(--err)">${esc(String(e.last_error).slice(0, 90))}</span>` : ""}</div>
</div>`;
}).join("") : '<div class="dim">暂无池内记录 — 开启预热并等待下一轮巡检</div>';
};
renderPool(ov);
overviewTimer = setInterval(() => { fetchOverview().then(renderPool).catch(() => {}); }, 5000);
$$("[data-rf]").forEach(b => b.onclick = async () => {
b.disabled = true;
try { await api("/api/actions", { body: { action: "refresh_cookies", site: b.dataset.rf } }); toast("已作废,将在下次请求时重新求解", "ok"); PAGES.turnstile(); }
try { await api("/api/actions", { body: { action: "refresh_cookies", site: b.dataset.rf } }); toast(t.warm_enabled ? "已作废,Cookie 池将在下一轮巡检时重新预热" : "已作废,将在下次请求时重新求解", "ok"); PAGES.turnstile(); }
catch (e) { toast(e.message, "err"); }
b.disabled = false;
});
$("#ts-provider").value = t.provider || "api";
$("#ts-style").value = t.api_style === "ezsolver" ? "ezsolver" : "sync";
const TS_HINTS = {
api: ["已开启:调用求解服务 API 获取 Turnstile Token", "已关闭:跳过 Turnstile 求解,直接请求上游站点"],
browser: ["已开启:用下方配置的本地浏览器自动求解 Token", "已关闭:跳过 Turnstile 求解,直接请求上游站点"],
@@ -521,6 +706,13 @@ PAGES.turnstile = async () => {
api_url: $("#ts-url").value.trim(),
api_key: $("#ts-key").value.trim(), sitekey: $("#ts-sitekey").value.trim(),
challenge_action: $("#ts-action").value.trim(), cookie_ttl_seconds: +$("#ts-ttl").value,
api_style: $("#ts-style").value,
timeout_seconds: Math.round(+$("#ts-timeout").value || 90),
retries: Math.round(+$("#ts-retries").value || 5),
retry_backoff_seconds: Math.max(0, +$("#ts-backoff").value || 0),
warm_enabled: $("#ts-warm").checked,
warm_ratio: Math.min(95, Math.max(5, Math.round(+$("#ts-ratio").value || 30))) / 100,
warm_check_seconds: Math.round(+$("#ts-check").value || 60),
browser_path: $("#ts-bpath").value.trim(),
browser_headless: $("#ts-bhead").checked,
browser_user_data_dir: $("#ts-bprof").value.trim(),
+1
View File
@@ -83,6 +83,7 @@ label{display:block;font-size:12px;color:var(--muted);margin-bottom:5px}
.pill{display:inline-block;padding:2px 8px;border-radius:20px;font-size:11px;border:1px solid var(--line);color:var(--muted)}
.pill.on{color:var(--ok);border-color:#1f6a34;background:#0f2318}
.pill.off{color:var(--dim)}
.pill.warn{color:var(--warn);border-color:#6b5417;background:#2a2210}
/* console / logs */
.log-box{background:#0a0e14;border:1px solid var(--line);border-radius:10px;padding:10px;height:56vh;overflow:auto;font-family:var(--mono);font-size:12.5px;line-height:1.55}
+230 -2
View File
@@ -14,6 +14,15 @@ import (
const DefaultUserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/149.0.0.0 Safari/537.36"
// Turnstile tuning defaults. DefaultSolveTimeoutSeconds is the single source
// for how long one solve attempt may run; the warm_* values drive the cookie
// pool (background pre-solving).
const (
DefaultSolveTimeoutSeconds = 90
DefaultWarmRatio = 0.3
DefaultWarmCheckSeconds = 60
)
// Turnstile token providers: api calls a hosted solver, browser drives a
// local Chrome/Edge over CDP, manual only ever uses imported cookies.
const (
@@ -29,11 +38,18 @@ type Turnstile struct {
APIKey string `toml:"api_key" json:"api_key"`
SiteKey string `toml:"sitekey" json:"sitekey"`
Action string `toml:"action" json:"action"`
APIStyle string `toml:"api_style" json:"api_style"`
TimeoutSeconds int `toml:"timeout_seconds" json:"timeout_seconds"`
CookieTTLSeconds int `toml:"cookie_ttl_seconds" json:"cookie_ttl_seconds"`
Retries int `toml:"retries" json:"retries"`
RetryBackoffSeconds float64 `toml:"retry_backoff_seconds" json:"retry_backoff_seconds"`
// Cookie pool: a background loop re-solves a site × route slot before its
// cached cookie expires, so requests never pay the solve latency.
WarmEnabled bool `toml:"warm_enabled" json:"warm_enabled"`
WarmRatio float64 `toml:"warm_ratio" json:"warm_ratio"`
WarmCheckSeconds int `toml:"warm_check_seconds" json:"warm_check_seconds"`
// provider = "browser"
BrowserPath string `toml:"browser_path" json:"browser_path"`
BrowserHeadless bool `toml:"browser_headless" json:"browser_headless"`
@@ -51,6 +67,112 @@ func (t Turnstile) ProviderValue() string {
return p
}
// APIStyleValue normalizes api_style for provider "api": "sync" (CapSolver-
// shaped single POST) or "ezsolver" (POST {sitekey,siteurl,timeout} → {token}).
func (t Turnstile) APIStyleValue() string {
switch strings.ToLower(strings.TrimSpace(t.APIStyle)) {
case "ezsolver", "ez":
return "ezsolver"
default:
return "sync"
}
}
// SolveTimeoutValue normalizes timeout_seconds: the longest one solve attempt
// may run (default 90s, clamped to 5–600s).
func (t Turnstile) SolveTimeoutValue() int {
n := t.TimeoutSeconds
if n <= 0 {
return DefaultSolveTimeoutSeconds
}
if n < 5 {
return 5
}
if n > 600 {
return 600
}
return n
}
// WarmRatioValue normalizes warm_ratio: a pooled cookie is re-solved once its
// remaining TTL drops below this fraction (default 30%, clamped 5–95%).
func (t Turnstile) WarmRatioValue() float64 {
r := t.WarmRatio
if r <= 0 {
return DefaultWarmRatio
}
if r < 0.05 {
return 0.05
}
if r > 0.95 {
return 0.95
}
return r
}
// WarmCheckValue normalizes warm_check_seconds: how often the pool is
// inspected (default 60s, clamped 5s–1h).
func (t Turnstile) WarmCheckValue() int {
n := t.WarmCheckSeconds
if n <= 0 {
return DefaultWarmCheckSeconds
}
if n < 5 {
return 5
}
if n > 3600 {
return 3600
}
return n
}
// Quota sentinel defaults: polling interval and the remaining-quota warning
// threshold (share of the daily free limit left).
const (
DefaultQuotaCheckSeconds = 300
DefaultQuotaWarnRatio = 0.2
)
// QuotaWatch configures the guest-token balance sentinel: it polls each
// site's balance endpoint with the pooled session (same egress and cookies as
// chat) and reports remaining daily quota to the console and logs.
type QuotaWatch struct {
Enabled bool `toml:"enabled" json:"enabled"`
CheckSeconds int `toml:"check_seconds" json:"check_seconds"`
WarnRatio float64 `toml:"warn_ratio" json:"warn_ratio"`
}
// CheckSecondsValue normalizes check_seconds (default 300s, clamped 30–3600).
func (q QuotaWatch) CheckSecondsValue() int {
n := q.CheckSeconds
if n <= 0 {
return DefaultQuotaCheckSeconds
}
if n < 30 {
return 30
}
if n > 3600 {
return 3600
}
return n
}
// WarnRatioValue normalizes warn_ratio: remaining/limit below this fraction
// raises a "quota low" warning (default 20%, clamped 5–95%).
func (q QuotaWatch) WarnRatioValue() float64 {
r := q.WarnRatio
if r <= 0 {
return DefaultQuotaWarnRatio
}
if r < 0.05 {
return 0.05
}
if r > 0.95 {
return 0.95
}
return r
}
type Site struct {
Code string `toml:"-"`
BaseURL string `toml:"base_url" json:"base_url"`
@@ -59,6 +181,41 @@ type Site struct {
VerifyAction string `toml:"verify_action" json:"verify_action"`
Language string `toml:"language" json:"language"`
Enabled bool `toml:"enabled" json:"enabled"`
// Proxies binds this site to the proxy pool: an ordered list of entry
// names and/or "sub:<name>" references. Empty = global [proxy] routes.
Proxies []string `toml:"proxies" json:"proxies"`
// WarmPoolOnly keeps the cookie warmer on the pool routes bound via
// Proxies; the global [proxy] primary/fallback lanes are not pre-solved
// (they remain available for request-time failover). Default false.
WarmPoolOnly bool `toml:"warm_pool_only" json:"warm_pool_only"`
}
// ProxyEntry is one manually configured pool node: an Xray share link
// (vless/vmess/trojan/ss) or a plain http/https/socks5 endpoint.
type ProxyEntry struct {
Link string `toml:"link" json:"link"`
Enabled bool `toml:"enabled" json:"enabled"`
}
// ProxySubscription is a remote node list refreshed on an interval.
type ProxySubscription struct {
URL string `toml:"url" json:"url"`
Enabled bool `toml:"enabled" json:"enabled"`
IntervalMinutes int `toml:"interval_minutes" json:"interval_minutes"`
}
// ProxyPool configures the egress pool and its managed Xray core.
type ProxyPool struct {
Enabled bool `toml:"enabled" json:"enabled"`
CheckIntervalSeconds int `toml:"check_interval_seconds" json:"check_interval_seconds"`
CheckTimeoutSeconds int `toml:"check_timeout_seconds" json:"check_timeout_seconds"`
CheckURL string `toml:"check_url" json:"check_url"`
DefaultScheme string `toml:"default_scheme" json:"default_scheme"`
XrayPath string `toml:"xray_path" json:"xray_path"`
XrayVersion string `toml:"xray_version" json:"xray_version"`
XrayAutoDownload bool `toml:"xray_auto_download" json:"xray_auto_download"`
Entries map[string]*ProxyEntry `toml:"entries" json:"entries"`
Subscriptions map[string]*ProxySubscription `toml:"subscriptions" json:"subscriptions"`
}
type Model struct {
@@ -77,6 +234,8 @@ type Config struct {
Host string `toml:"host" json:"host"`
Port int `toml:"port" json:"port"`
LogLevel string `toml:"log_level" json:"log_level"`
// LogFile: "" = <data_dir>/logs/dsfree2api.log, "-" = stderr only.
LogFile string `toml:"log_file" json:"log_file"`
} `toml:"server" json:"server"`
Security struct {
@@ -101,6 +260,12 @@ type Config struct {
SlowStartSeconds float64 `toml:"slow_start_seconds" json:"slow_start_seconds"`
} `toml:"proxy" json:"proxy"`
ProxyPool ProxyPool `toml:"proxypool" json:"proxypool"`
// Quota is the balance sentinel: it polls each site's guest-token balance
// so the console shows how much daily free quota is left.
Quota QuotaWatch `toml:"quota" json:"quota"`
Upstream struct {
Timeout float64 `toml:"timeout" json:"timeout"`
StreamTimeout float64 `toml:"stream_timeout" json:"stream_timeout"`
@@ -125,6 +290,20 @@ type Config struct {
path string `toml:"-"`
}
// DefaultProxyPool returns the built-in pool settings (feature off, Xray
// auto-download on).
func DefaultProxyPool() ProxyPool {
return ProxyPool{
CheckIntervalSeconds: 120,
CheckTimeoutSeconds: 10,
CheckURL: "https://www.gstatic.com/generate_204",
DefaultScheme: "http",
XrayAutoDownload: true,
Entries: map[string]*ProxyEntry{},
Subscriptions: map[string]*ProxySubscription{},
}
}
func DefaultTurnstile() Turnstile {
return Turnstile{
Enabled: false,
@@ -133,10 +312,12 @@ func DefaultTurnstile() Turnstile {
APIKey: "",
SiteKey: "0x4AAAAAADlLZ3ljqZP6cQwq",
Action: "chat",
TimeoutSeconds: 90,
TimeoutSeconds: DefaultSolveTimeoutSeconds,
CookieTTLSeconds: 10800,
Retries: 5,
RetryBackoffSeconds: 1.5,
WarmRatio: DefaultWarmRatio,
WarmCheckSeconds: DefaultWarmCheckSeconds,
}
}
@@ -201,6 +382,7 @@ func newConfig() *Config {
c.Upstream.CrossSiteFailover = true
c.Upstream.ContinueRounds = 20
c.Runtime.DataDir = "./data"
c.ProxyPool = DefaultProxyPool()
c.Turnstile = DefaultTurnstile()
c.Sites = DefaultSites(c.Turnstile.SiteKey)
c.Models = DefaultModels()
@@ -300,22 +482,36 @@ func (c *Config) applyDefaults(md *toml.MetaData) {
c.Turnstile.Enabled = true
c.Upstream.AutoRefresh = true
c.Upstream.ContinueRounds = 20
c.ProxyPool.XrayAutoDownload = true
for _, s := range c.Sites {
s.Enabled = true
}
for _, m := range c.Models {
m.Enabled = true
}
for _, e := range c.ProxyPool.Entries {
e.Enabled = true
}
for _, sc := range c.ProxyPool.Subscriptions {
sc.Enabled = true
}
return
}
def(md.IsDefined("turnstile", "enabled"), &c.Turnstile.Enabled, true)
def(md.IsDefined("upstream", "auto_refresh"), &c.Upstream.AutoRefresh, true)
def(md.IsDefined("proxypool", "xray_auto_download"), &c.ProxyPool.XrayAutoDownload, true)
for code, s := range c.Sites {
def(md.IsDefined("sites", code, "enabled"), &s.Enabled, true)
}
for id, m := range c.Models {
def(md.IsDefined("models", id, "enabled"), &m.Enabled, true)
}
for name, e := range c.ProxyPool.Entries {
def(md.IsDefined("proxypool", "entries", name, "enabled"), &e.Enabled, true)
}
for name, sc := range c.ProxyPool.Subscriptions {
def(md.IsDefined("proxypool", "subscriptions", name, "enabled"), &sc.Enabled, true)
}
}
func (c *Config) applyEnv() error {
@@ -331,6 +527,9 @@ func (c *Config) applyEnv() error {
if v := str("LOG_LEVEL"); v != nil {
c.Server.LogLevel = *v
}
if v := str("LOG_FILE"); v != nil {
c.Server.LogFile = strings.TrimSpace(*v)
}
if v := str("PORT"); v != nil {
n, err := strconv.Atoi(strings.TrimSpace(*v))
if err != nil || n < 1 || n > 65535 {
@@ -363,6 +562,34 @@ func (c *Config) applyEnv() error {
if v := str("TURNSTILE_BROWSER_PATH"); v != nil && strings.TrimSpace(*v) != "" {
c.Turnstile.BrowserPath = strings.TrimSpace(*v)
}
if v := str("TURNSTILE_TIMEOUT_SECONDS"); v != nil {
n, err := strconv.Atoi(strings.TrimSpace(*v))
if err != nil || n < 1 {
return fmt.Errorf("invalid TURNSTILE_TIMEOUT_SECONDS %q", *v)
}
c.Turnstile.TimeoutSeconds = n
}
if v := str("TURNSTILE_WARM_ENABLED"); v != nil {
b, err := strconv.ParseBool(strings.TrimSpace(*v))
if err != nil {
return fmt.Errorf("invalid TURNSTILE_WARM_ENABLED %q", *v)
}
c.Turnstile.WarmEnabled = b
}
if v := str("TURNSTILE_WARM_RATIO"); v != nil {
f, err := strconv.ParseFloat(strings.TrimSpace(*v), 64)
if err != nil || f <= 0 || f > 1 {
return fmt.Errorf("invalid TURNSTILE_WARM_RATIO %q (expected 0–1)", *v)
}
c.Turnstile.WarmRatio = f
}
if v := str("TURNSTILE_WARM_CHECK_SECONDS"); v != nil {
n, err := strconv.Atoi(strings.TrimSpace(*v))
if err != nil || n < 1 {
return fmt.Errorf("invalid TURNSTILE_WARM_CHECK_SECONDS %q", *v)
}
c.Turnstile.WarmCheckSeconds = n
}
if v := str("TURNSTILE_ENABLED"); v != nil {
b, err := strconv.ParseBool(strings.TrimSpace(*v))
if err != nil {
@@ -452,7 +679,8 @@ func (c *Config) validateTurnstile() error {
if c.Turnstile.APIURL == "" {
return errors.New("turnstile.api_url is empty (set config to your solver endpoint, or use provider = \"manual\")")
}
if c.Turnstile.APIKey == "" {
// Local solve services (api_style = "ezsolver") usually need no key.
if c.Turnstile.APIKey == "" && c.Turnstile.APIStyleValue() != "ezsolver" {
return errors.New("turnstile.api_key is empty (set config, or TURNSTILE_API_KEY)")
}
case ProviderBrowser:
+77
View File
@@ -100,6 +100,11 @@ func TestValidateRejectsBadValues(t *testing.T) {
if err := cfg.Validate(); err == nil {
t.Error("expected error for empty turnstile api_key")
}
cfg.Turnstile.APIStyle = "ezsolver"
if err := cfg.Validate(); err != nil {
t.Errorf("ezsolver style without api_key: %v", err)
}
cfg.Turnstile.APIStyle = ""
cfg.Turnstile.APIKey = "k"
cfg.Turnstile.Enabled = false
cfg.Admin.Port = cfg.Server.Port
@@ -204,6 +209,78 @@ func TestTurnstileProviderEnv(t *testing.T) {
}
}
func TestProxyPoolDefaults(t *testing.T) {
d := DefaultProxyPool()
if d.Enabled {
t.Error("proxy pool must be opt-in")
}
if !d.XrayAutoDownload || d.CheckIntervalSeconds != 120 || d.CheckTimeoutSeconds != 10 {
t.Errorf("pool defaults = %+v", d)
}
if d.DefaultScheme != "http" {
t.Errorf("default scheme = %q", d.DefaultScheme)
}
cfg, err := Load(examplePath)
if err != nil {
t.Fatalf("load: %v", err)
}
if !cfg.ProxyPool.XrayAutoDownload {
t.Error("example config should keep xray_auto_download = true")
}
if cfg.ProxyPool.CheckIntervalSeconds != 120 {
t.Errorf("check interval = %d", cfg.ProxyPool.CheckIntervalSeconds)
}
}
func TestTurnstileSolveTimeoutAndWarmDefaults(t *testing.T) {
d := DefaultTurnstile()
if d.WarmEnabled {
t.Error("cookie pool must be opt-in")
}
if d.WarmRatio != 0.3 || d.WarmCheckSeconds != 60 {
t.Errorf("warm defaults = %v / %v", d.WarmRatio, d.WarmCheckSeconds)
}
z := Turnstile{}
if z.SolveTimeoutValue() != 90 {
t.Errorf("zero timeout should normalize to 90, got %d", z.SolveTimeoutValue())
}
if z.WarmRatioValue() != 0.3 || z.WarmCheckValue() != 60 {
t.Errorf("zero warm values should normalize to 0.3 / 60")
}
if got := (Turnstile{TimeoutSeconds: 1}).SolveTimeoutValue(); got != 5 {
t.Errorf("timeout floor = %d, want 5", got)
}
if got := (Turnstile{TimeoutSeconds: 9999}).SolveTimeoutValue(); got != 600 {
t.Errorf("timeout ceiling = %d, want 600", got)
}
if got := (Turnstile{WarmRatio: 0.99}).WarmRatioValue(); got != 0.95 {
t.Errorf("warm ratio ceiling = %v, want 0.95", got)
}
if (Turnstile{}).APIStyleValue() != "sync" {
t.Error("zero api_style should normalize to sync")
}
if (Turnstile{APIStyle: "EzSolver"}).APIStyleValue() != "ezsolver" {
t.Error("api_style should normalize EzSolver")
}
}
func TestTurnstileSolveAndWarmEnvOverrides(t *testing.T) {
t.Setenv("TURNSTILE_TIMEOUT_SECONDS", "45")
t.Setenv("TURNSTILE_WARM_ENABLED", "true")
t.Setenv("TURNSTILE_WARM_RATIO", "0.5")
t.Setenv("TURNSTILE_WARM_CHECK_SECONDS", "120")
cfg, err := Load(examplePath)
if err != nil {
t.Fatalf("load: %v", err)
}
if cfg.Turnstile.TimeoutSeconds != 45 {
t.Errorf("timeout = %d, want 45", cfg.Turnstile.TimeoutSeconds)
}
if !cfg.Turnstile.WarmEnabled || cfg.Turnstile.WarmRatio != 0.5 || cfg.Turnstile.WarmCheckSeconds != 120 {
t.Errorf("warm env = %+v", cfg.Turnstile)
}
}
func TestSaveRoundTrip(t *testing.T) {
cfg, err := Load(examplePath)
if err != nil {
+170
View File
@@ -0,0 +1,170 @@
// Package logfile writes logs to a daily file and compresses the previous
// day's file on rotation. It is an io.Writer meant to sit behind an slog
// handler, next to the stderr writer.
package logfile
import (
"compress/gzip"
"fmt"
"io"
"os"
"path/filepath"
"strings"
"sync"
"time"
)
const dayLayout = "2006-01-02"
// Writer appends to path and rotates once per day: the retired file becomes
// "<base>-YYYY-MM-DD.log.gz" next to it.
type Writer struct {
mu sync.Mutex
dir string
base string
path string
file *os.File
day string
closed bool
now func() time.Time
wg sync.WaitGroup
}
// New opens (or creates) path and immediately rotates a stale file left over
// from an earlier day.
func New(path string) (*Writer, error) {
path = strings.TrimSpace(path)
if path == "" {
return nil, fmt.Errorf("log file path is empty")
}
dir := filepath.Dir(path)
if err := os.MkdirAll(dir, 0o755); err != nil {
return nil, err
}
w := &Writer{dir: dir, base: baseName(path), path: path, now: time.Now}
if st, err := os.Stat(path); err == nil {
day := st.ModTime().Format(dayLayout)
if day != w.now().Format(dayLayout) && st.Size() > 0 {
w.day = day
if err := w.rotateLocked(); err != nil {
return nil, err
}
return w, nil
}
}
return w, w.openLocked()
}
// Write appends p, rotating first when the local day changed.
func (w *Writer) Write(p []byte) (int, error) {
w.mu.Lock()
defer w.mu.Unlock()
if w.closed {
return 0, os.ErrClosed
}
if w.file == nil {
if err := w.openLocked(); err != nil {
return 0, err
}
}
if day := w.now().Format(dayLayout); day != w.day {
if err := w.rotateLocked(); err != nil {
return 0, err
}
}
return w.file.Write(p)
}
// Close flushes and waits for any in-flight compression.
func (w *Writer) Close() error {
w.mu.Lock()
defer w.mu.Unlock()
if w.closed {
return nil
}
w.closed = true
var err error
if w.file != nil {
err = w.file.Close()
w.file = nil
}
w.wg.Wait()
return err
}
func (w *Writer) openLocked() error {
f, err := os.OpenFile(w.path, os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0o644)
if err != nil {
return err
}
w.file = f
w.day = w.now().Format(dayLayout)
return nil
}
// rotateLocked retires the current file — renaming it first so the async
// compression can never read a freshly opened file — and starts a new one.
func (w *Writer) rotateLocked() error {
if w.file != nil {
_ = w.file.Close()
w.file = nil
}
old, day := w.path, w.day
if st, err := os.Stat(old); err == nil && st.Size() > 0 {
tmp := old + ".rotating"
if err := os.Rename(old, tmp); err == nil {
gz := filepath.Join(w.dir, fmt.Sprintf("%s-%s.log.gz", w.base, day))
w.wg.Add(1)
go func() {
defer w.wg.Done()
compressFile(tmp, gz)
}()
}
} else {
_ = os.Remove(old)
}
return w.openLocked()
}
// compressFile gzips src into dst and removes src on success; on any failure
// the raw .rotating file is left in place so no log data is ever lost.
// NOTE: the source handle is closed before removal — Windows refuses to
// delete a file that is still open.
func compressFile(src, dst string) {
in, err := os.Open(src)
if err != nil {
return
}
out, err := os.Create(dst)
if err != nil {
_ = in.Close()
return
}
gz := gzip.NewWriter(out)
_, copyErr := io.Copy(gz, in)
_ = in.Close()
if copyErr != nil {
gz.Close()
out.Close()
_ = os.Remove(dst)
return
}
if err := gz.Close(); err != nil {
out.Close()
_ = os.Remove(dst)
return
}
if err := out.Close(); err != nil {
_ = os.Remove(dst)
return
}
_ = os.Remove(src)
}
func baseName(path string) string {
base := filepath.Base(path)
if ext := filepath.Ext(base); ext != "" {
base = strings.TrimSuffix(base, ext)
}
return base
}
+113
View File
@@ -0,0 +1,113 @@
package logfile
import (
"compress/gzip"
"io"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
func readGz(t *testing.T, path string) string {
t.Helper()
f, err := os.Open(path)
if err != nil {
t.Fatalf("open %s: %v", path, err)
}
defer f.Close()
gz, err := gzip.NewReader(f)
if err != nil {
t.Fatalf("gzip %s: %v", path, err)
}
defer gz.Close()
b, err := io.ReadAll(gz)
if err != nil {
t.Fatalf("read %s: %v", path, err)
}
return string(b)
}
func TestDailyRotationCompresses(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "dsfree2api.log")
w, err := New(path)
if err != nil {
t.Fatalf("new: %v", err)
}
day1 := w.now()
if _, err := w.Write([]byte("day-one\n")); err != nil {
t.Fatalf("write: %v", err)
}
// Cross midnight: the next write rotates and compresses day one.
w.now = func() time.Time { return day1.Add(24 * time.Hour) }
if _, err := w.Write([]byte("day-two\n")); err != nil {
t.Fatalf("write: %v", err)
}
if err := w.Close(); err != nil {
t.Fatalf("close: %v", err)
}
cur, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read current: %v", err)
}
if !strings.Contains(string(cur), "day-two") || strings.Contains(string(cur), "day-one") {
t.Fatalf("current log = %q", cur)
}
gzPath := filepath.Join(dir, "dsfree2api-"+day1.Format(dayLayout)+".log.gz")
if got := readGz(t, gzPath); !strings.Contains(got, "day-one") {
t.Fatalf("rotated log = %q", got)
}
if _, err := os.Stat(path + ".rotating"); !os.IsNotExist(err) {
t.Error(".rotating file should be gone after compression")
}
}
func TestWriteAfterCloseFails(t *testing.T) {
w, err := New(filepath.Join(t.TempDir(), "x.log"))
if err != nil {
t.Fatalf("new: %v", err)
}
if err := w.Close(); err != nil {
t.Fatalf("close: %v", err)
}
if _, err := w.Write([]byte("no\n")); err == nil {
t.Fatal("write after close should fail")
}
}
func TestStaleFileRotatedOnOpen(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "app.log")
if err := os.WriteFile(path, []byte("old\n"), 0o644); err != nil {
t.Fatalf("seed: %v", err)
}
old := time.Now().Add(-48 * time.Hour)
if err := os.Chtimes(path, old, old); err != nil {
t.Fatalf("chtimes: %v", err)
}
w, err := New(path)
if err != nil {
t.Fatalf("new: %v", err)
}
if _, err := w.Write([]byte("new\n")); err != nil {
t.Fatalf("write: %v", err)
}
if err := w.Close(); err != nil {
t.Fatalf("close: %v", err)
}
gzPath := filepath.Join(dir, "app-"+old.Format(dayLayout)+".log.gz")
if got := readGz(t, gzPath); !strings.Contains(got, "old") {
t.Fatalf("rotated = %q", got)
}
cur, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read current: %v", err)
}
if !strings.Contains(string(cur), "new") {
t.Fatalf("current = %q", cur)
}
}
+46 -3
View File
@@ -46,9 +46,11 @@ type modelStats struct {
}
type siteStats struct {
Requests int64 `json:"requests"`
Errors int64 `json:"errors"`
Quota int64 `json:"quota_exhausted"`
Requests int64 `json:"requests"`
Errors int64 `json:"errors"`
Quota int64 `json:"quota_exhausted"`
Prompt int64 `json:"prompt_tokens"`
Completed int64 `json:"completion_tokens"`
}
type dayStats struct {
@@ -228,6 +230,8 @@ func (r *Recorder) Record(rec Record) {
r.sites[rec.Site] = s
}
s.Requests++
s.Prompt += int64(rec.PromptTokens)
s.Completed += int64(rec.CompletionTokens)
if rec.Status == StatusError || rec.Status == StatusEmpty {
s.Errors++
}
@@ -264,6 +268,45 @@ func (r *Recorder) Record(rec Record) {
}
}
// SiteTotals is the cumulative per-site usage snapshot; it sizes a quota
// exhaustion — how many tokens had flowed before the site cut us off.
type SiteTotals struct {
Requests int64 `json:"requests"`
PromptTokens int64 `json:"prompt_tokens"`
CompletionTokens int64 `json:"completion_tokens"`
Quota int64 `json:"quota_exhausted"`
}
// SiteTotals returns the cumulative usage attributed to one site so far.
// Totals persist across restarts through metrics.json.
func (r *Recorder) SiteTotals(site string) SiteTotals {
r.mu.Lock()
defer r.mu.Unlock()
s := r.sites[site]
if s == nil {
return SiteTotals{}
}
return SiteTotals{
Requests: s.Requests,
PromptTokens: s.Prompt,
CompletionTokens: s.Completed,
Quota: s.Quota,
}
}
// IncSiteQuota counts one site-side quota exhaustion that may never surface as
// a failed request (e.g. swallowed by cross-site failover).
func (r *Recorder) IncSiteQuota(site string) {
r.mu.Lock()
defer r.mu.Unlock()
s := r.sites[site]
if s == nil {
s = &siteStats{}
r.sites[site] = s
}
s.Quota++
}
func (r *Recorder) Snapshot() any {
r.mu.Lock()
defer r.mu.Unlock()
+26
View File
@@ -0,0 +1,26 @@
package metrics
import "testing"
func TestRecordTracksSiteTokensAndQuota(t *testing.T) {
r := New(t.TempDir())
r.Record(Record{Model: "deepseek-v4-flash-de", Site: "de", PromptTokens: 120, CompletionTokens: 80, Status: StatusOK})
r.Record(Record{Model: "deepseek-v4-flash-de", Site: "de", PromptTokens: 15, Status: StatusError, Error: "sse quota exhausted"})
got := r.SiteTotals("de")
if got.Requests != 2 {
t.Fatalf("Requests = %d, want 2", got.Requests)
}
if got.PromptTokens != 135 || got.CompletionTokens != 80 {
t.Errorf("tokens = %d/%d, want 135/80", got.PromptTokens, got.CompletionTokens)
}
r.IncSiteQuota("de")
if got := r.SiteTotals("de"); got.Quota != 1 {
t.Errorf("Quota = %d, want 1", got.Quota)
}
if empty := r.SiteTotals("fr"); empty.Requests != 0 || empty.Quota != 0 {
t.Errorf("unknown site totals = %+v, want zero", empty)
}
}
+722
View File
@@ -0,0 +1,722 @@
package proxypool
import (
"context"
"crypto/sha256"
"encoding/hex"
"fmt"
"io"
"log/slog"
"net/http"
"net/url"
"sort"
"strings"
"sync"
"time"
"github.com/nyoungo/dsfree2api/internal/config"
"github.com/nyoungo/dsfree2api/internal/upstream"
)
// endpoint is one runtime pool slot.
type endpoint struct {
name string
kind string // "xray" | "url" | "invalid"
scheme string
display string
link string
source string // "" = manual entry, else subscription name
enabled bool
parseErr string
outbound map[string]any
url string
proxyURL string // resolved egress URL (socks5://127.0.0.1:port or plain)
checked bool
healthy bool
latencyMs int64
lastErr string
checkedAt time.Time
}
type subState struct {
name string
nodes []string
lastErr string
fetchedAt time.Time
}
// Manager owns the proxy pool: endpoints, health, subscriptions and sticky
// per-site selection. It is safe for concurrent use.
type Manager struct {
cfg *config.Config
log *slog.Logger
dir string
xray *xrayCore
reloadMu sync.Mutex
mu sync.Mutex
endpoints map[string]*endpoint
subs map[string]*subState
sticky map[string]string
fetching map[string]bool
started bool
lastCheck time.Time
lastErr string
}
func New(cfg *config.Config, dataDir string, log *slog.Logger) *Manager {
if log == nil {
log = slog.Default()
}
return &Manager{
cfg: cfg,
log: log,
dir: dataDir,
xray: newXrayCore(dataDir, log),
endpoints: map[string]*endpoint{},
subs: map[string]*subState{},
sticky: map[string]string{},
fetching: map[string]bool{},
}
}
// Start runs the background loop: reload from config, refresh due
// subscriptions, probe endpoint health.
func (m *Manager) Start(ctx context.Context) {
m.mu.Lock()
if m.started {
m.mu.Unlock()
return
}
m.started = true
m.mu.Unlock()
go m.loop(ctx)
}
func (m *Manager) loop(ctx context.Context) {
ticker := time.NewTicker(5 * time.Second)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
m.xray.stop()
return
case <-ticker.C:
}
m.cycle(ctx)
}
}
func (m *Manager) cycle(ctx context.Context) {
m.Reload()
m.cfg.RLock()
enabled := m.cfg.ProxyPool.Enabled
interval := time.Duration(m.cfg.ProxyPool.CheckIntervalSeconds) * time.Second
subs := make(map[string]config.ProxySubscription, len(m.cfg.ProxyPool.Subscriptions))
for name, sc := range m.cfg.ProxyPool.Subscriptions {
if sc != nil {
subs[name] = *sc
}
}
m.cfg.RUnlock()
if !enabled {
return
}
if interval <= 0 {
interval = 120 * time.Second
}
m.mu.Lock()
due := m.lastCheck.IsZero() || time.Since(m.lastCheck) >= interval
m.mu.Unlock()
if due {
m.checkAll(ctx)
}
now := time.Now()
for name, sc := range subs {
if !sc.Enabled || strings.TrimSpace(sc.URL) == "" {
continue
}
m.mu.Lock()
st := m.subs[name]
busy := m.fetching[name]
m.mu.Unlock()
if busy {
continue
}
iv := time.Duration(sc.IntervalMinutes) * time.Minute
if iv <= 0 {
iv = time.Hour
}
if st != nil && !st.fetchedAt.IsZero() && now.Sub(st.fetchedAt) < iv {
continue
}
go m.fetchSub(ctx, name, sc)
}
}
// Reload rebuilds the runtime endpoint table from the live config plus the
// cached subscription nodes, (re)configures the Xray core and prunes sticky
// picks that no longer resolve.
func (m *Manager) Reload() {
m.reloadMu.Lock()
defer m.reloadMu.Unlock()
m.cfg.RLock()
poolCfg := m.cfg.ProxyPool
entries := make(map[string]config.ProxyEntry, len(poolCfg.Entries))
for name, e := range poolCfg.Entries {
if e != nil {
entries[name] = *e
}
}
subsCfg := make(map[string]config.ProxySubscription, len(poolCfg.Subscriptions))
for name, sc := range poolCfg.Subscriptions {
if sc != nil {
subsCfg[name] = *sc
}
}
m.cfg.RUnlock()
m.mu.Lock()
subNodes := make(map[string][]string, len(m.subs))
for name, st := range m.subs {
subNodes[name] = append([]string(nil), st.nodes...)
}
old := m.endpoints
m.mu.Unlock()
desired := map[string]*endpoint{}
for name, e := range entries {
if strings.TrimSpace(e.Link) == "" {
continue
}
ep := newEndpoint(name, e.Link, poolCfg.DefaultScheme, "")
ep.enabled = e.Enabled
desired[name] = ep
}
for subName, nodes := range subNodes {
sc, ok := subsCfg[subName]
if !ok {
continue
}
for _, line := range nodes {
name := subName + "#" + shortHash(line)
if _, dup := desired[name]; dup {
continue
}
ep := newEndpoint(name, line, poolCfg.DefaultScheme, subName)
ep.enabled = sc.Enabled
desired[name] = ep
}
}
for name, ep := range desired {
if o := old[name]; o != nil && o.link == ep.link {
ep.checked = o.checked
ep.healthy = o.healthy
ep.latencyMs = o.latencyMs
ep.lastErr = o.lastErr
ep.checkedAt = o.checkedAt
}
}
nodes := make([]xrayNode, 0, len(desired))
for _, ep := range desired {
if ep.kind == "xray" && ep.enabled && ep.parseErr == "" && ep.outbound != nil {
nodes = append(nodes, xrayNode{Name: ep.name, Outbound: ep.outbound})
}
}
sort.Slice(nodes, func(i, j int) bool { return nodes[i].Name < nodes[j].Name })
if len(nodes) > 0 {
if err := m.xray.ensure(poolCfg.XrayPath, poolCfg.XrayAutoDownload, poolCfg.XrayVersion); err != nil {
m.setErr(err)
} else if err := m.xray.configure(nodes); err != nil {
m.setErr(err)
} else {
m.setErr(nil)
}
} else {
_ = m.xray.configure(nil)
m.setErr(nil)
}
xrayUp := m.xray.status().Running
for _, ep := range desired {
switch ep.kind {
case "url":
ep.proxyURL = ep.url
case "xray":
if ep.parseErr == "" && xrayUp {
if port := m.xray.port(ep.name); port > 0 {
ep.proxyURL = fmt.Sprintf("socks5://127.0.0.1:%d", port)
}
}
}
}
m.mu.Lock()
m.endpoints = desired
for site, name := range m.sticky {
ep := desired[name]
if ep == nil || !ep.enabled || ep.proxyURL == "" {
delete(m.sticky, site)
}
}
m.mu.Unlock()
}
// RefreshNow refreshes every subscription and probes every endpoint, then
// reloads. Used by the console button.
func (m *Manager) RefreshNow(ctx context.Context) {
go func() {
m.Reload()
m.cfg.RLock()
enabled := m.cfg.ProxyPool.Enabled
subs := make(map[string]config.ProxySubscription, len(m.cfg.ProxyPool.Subscriptions))
for name, sc := range m.cfg.ProxyPool.Subscriptions {
if sc != nil {
subs[name] = *sc
}
}
m.cfg.RUnlock()
for name, sc := range subs {
if sc.Enabled && strings.TrimSpace(sc.URL) != "" {
m.fetchSub(ctx, name, sc)
}
}
if enabled {
m.checkAll(ctx)
}
m.Reload()
}()
}
func (m *Manager) fetchSub(ctx context.Context, name string, sc config.ProxySubscription) {
m.mu.Lock()
m.fetching[name] = true
m.mu.Unlock()
defer func() {
m.mu.Lock()
delete(m.fetching, name)
m.mu.Unlock()
}()
client, err := m.fetchClient()
if err == nil {
var req *http.Request
req, err = http.NewRequestWithContext(ctx, http.MethodGet, sc.URL, nil)
if err == nil {
var resp *http.Response
resp, err = client.Do(req)
if err == nil {
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
err = fmt.Errorf("subscription http %d", resp.StatusCode)
} else {
var raw []byte
raw, err = io.ReadAll(io.LimitReader(resp.Body, 8<<20))
if err == nil {
lines := ParseSubscriptionBody(raw)
if len(lines) == 0 {
err = fmt.Errorf("subscription returned no nodes")
} else {
err = nil
m.subResult(name, lines, nil)
m.log.Info("subscription refreshed", "name", name, "nodes", len(lines))
m.Reload()
return
}
}
}
}
}
}
m.subResult(name, nil, err)
m.log.Warn("subscription refresh failed", "name", name, "error", err)
}
func (m *Manager) subResult(name string, nodes []string, err error) {
m.mu.Lock()
defer m.mu.Unlock()
st := m.subs[name]
if st == nil {
st = &subState{name: name}
m.subs[name] = st
}
if err != nil {
st.lastErr = err.Error()
return
}
st.nodes = nodes
st.lastErr = ""
st.fetchedAt = time.Now()
}
func (m *Manager) fetchClient() (*http.Client, error) {
m.cfg.RLock()
proxy := strings.TrimSpace(m.cfg.Proxy.URL)
m.cfg.RUnlock()
tr := &http.Transport{
TLSHandshakeTimeout: 20 * time.Second,
ResponseHeaderTimeout: 30 * time.Second,
}
if proxy != "" {
u, err := url.Parse(proxy)
if err != nil {
return nil, fmt.Errorf("proxy url: %w", err)
}
tr.Proxy = http.ProxyURL(u)
}
return &http.Client{Transport: tr, Timeout: 45 * time.Second}, nil
}
func (m *Manager) checkAll(ctx context.Context) {
m.mu.Lock()
list := make([]*endpoint, 0, len(m.endpoints))
for _, ep := range m.endpoints {
list = append(list, ep)
}
m.mu.Unlock()
m.cfg.RLock()
timeout := time.Duration(m.cfg.ProxyPool.CheckTimeoutSeconds) * time.Second
checkURL := strings.TrimSpace(m.cfg.ProxyPool.CheckURL)
m.cfg.RUnlock()
if timeout <= 0 {
timeout = 10 * time.Second
}
if checkURL == "" {
checkURL = "https://www.gstatic.com/generate_204"
}
sem := make(chan struct{}, 4)
var wg sync.WaitGroup
for _, ep := range list {
if !ep.enabled || ep.proxyURL == "" {
continue
}
wg.Add(1)
sem <- struct{}{}
go func(ep *endpoint) {
defer wg.Done()
defer func() { <-sem }()
if ctx.Err() != nil {
return
}
ok, ms, errMsg := probeURL(ctx, ep.proxyURL, checkURL, timeout)
m.mu.Lock()
ep.checked = true
ep.checkedAt = time.Now()
ep.healthy = ok
ep.latencyMs = ms
ep.lastErr = errMsg
m.mu.Unlock()
}(ep)
}
wg.Wait()
m.mu.Lock()
m.lastCheck = time.Now()
m.mu.Unlock()
}
func probeURL(ctx context.Context, proxyURL, checkURL string, timeout time.Duration) (bool, int64, string) {
u, err := url.Parse(proxyURL)
if err != nil {
return false, 0, "bad proxy url"
}
tr := &http.Transport{
Proxy: http.ProxyURL(u),
TLSHandshakeTimeout: timeout,
ResponseHeaderTimeout: timeout,
DisableKeepAlives: true,
}
client := &http.Client{Transport: tr, Timeout: timeout}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, checkURL, nil)
if err != nil {
return false, 0, err.Error()
}
start := time.Now()
resp, err := client.Do(req)
ms := time.Since(start).Milliseconds()
if err != nil {
return false, ms, trimErr(err)
}
defer resp.Body.Close()
_, _ = io.Copy(io.Discard, io.LimitReader(resp.Body, 4096))
if resp.StatusCode >= 400 {
return false, ms, fmt.Sprintf("http %d", resp.StatusCode)
}
return true, ms, ""
}
// Candidates resolves the ordered route candidates for one site: its bound
// entries (or subscriptions) filtered to healthy ones, sticky pick first.
// It implements upstream.RouteSource.
func (m *Manager) Candidates(site string) []upstream.RouteCandidate {
m.cfg.RLock()
enabled := m.cfg.ProxyPool.Enabled
var items []string
if st, ok := m.cfg.Sites[site]; ok && st != nil {
items = append(items, st.Proxies...)
}
m.cfg.RUnlock()
if !enabled || len(items) == 0 {
return nil
}
m.mu.Lock()
defer m.mu.Unlock()
var all []*endpoint
for _, item := range items {
all = append(all, m.resolveItemLocked(item)...)
}
usable := make([]*endpoint, 0, len(all))
for _, ep := range all {
if ep.enabled && ep.proxyURL != "" && ep.parseErr == "" {
usable = append(usable, ep)
}
}
pick := make([]*endpoint, 0, len(usable))
for _, ep := range usable {
if !ep.checked || ep.healthy {
pick = append(pick, ep)
}
}
if len(pick) == 0 {
pick = usable
}
if len(pick) == 0 {
return nil
}
if chosen := m.sticky[site]; chosen != "" {
for i, ep := range pick {
if ep.name == chosen && i > 0 {
pick = append([]*endpoint{ep}, append(pick[:i], pick[i+1:]...)...)
break
}
}
}
m.sticky[site] = pick[0].name
out := make([]upstream.RouteCandidate, 0, len(pick))
for _, ep := range pick {
out = append(out, upstream.RouteCandidate{Name: ep.name, Proxy: ep.proxyURL})
}
return out
}
func (m *Manager) resolveItemLocked(item string) []*endpoint {
item = strings.TrimSpace(item)
if item == "" {
return nil
}
if name, ok := strings.CutPrefix(item, "sub:"); ok {
var out []*endpoint
for _, ep := range m.endpoints {
if ep.source == name {
out = append(out, ep)
}
}
sort.Slice(out, func(i, j int) bool { return out[i].name < out[j].name })
return out
}
if ep := m.endpoints[item]; ep != nil {
return []*endpoint{ep}
}
return nil
}
// Report records how a request through `proxy` went, so a failing endpoint
// loses its sticky pick immediately and a healthy one is trusted again.
func (m *Manager) Report(proxy string, ok bool) {
if m == nil || proxy == "" {
return
}
m.mu.Lock()
defer m.mu.Unlock()
for _, ep := range m.endpoints {
if ep.proxyURL != proxy {
continue
}
ep.checked = true
ep.checkedAt = time.Now()
if ok {
ep.healthy = true
ep.lastErr = ""
} else {
ep.healthy = false
ep.lastErr = "request failed"
for site, name := range m.sticky {
if name == ep.name {
delete(m.sticky, site)
}
}
}
return
}
}
// Status is the console snapshot.
type Status struct {
Enabled bool `json:"enabled"`
CheckIntervalS int `json:"check_interval_seconds"`
CheckTimeoutS int `json:"check_timeout_seconds"`
CheckURL string `json:"check_url"`
DefaultScheme string `json:"default_scheme"`
XrayPath string `json:"xray_path"`
XrayVersion string `json:"xray_version"`
XrayAutoDownload bool `json:"xray_auto_download"`
Xray XrayStatus `json:"xray"`
Entries []EntryStatus `json:"entries"`
Subscriptions []SubStatus `json:"subscriptions"`
Sticky map[string]string `json:"sticky"`
LastError string `json:"last_error,omitempty"`
}
type EntryStatus struct {
Name string `json:"name"`
Kind string `json:"kind"`
Scheme string `json:"scheme"`
Target string `json:"target"`
Source string `json:"source,omitempty"`
Enabled bool `json:"enabled"`
Healthy bool `json:"healthy"`
Checked bool `json:"checked"`
LatencyMs int64 `json:"latency_ms"`
LastErr string `json:"last_error,omitempty"`
ParseError string `json:"parse_error,omitempty"`
CheckedAt int64 `json:"checked_at"`
}
type SubStatus struct {
Name string `json:"name"`
URL string `json:"url"`
Enabled bool `json:"enabled"`
IntervalMinutes int `json:"interval_minutes"`
Nodes int `json:"nodes"`
FetchedAt int64 `json:"fetched_at"`
LastErr string `json:"last_error,omitempty"`
}
func (m *Manager) Status() Status {
m.cfg.RLock()
poolCfg := m.cfg.ProxyPool
subsCfg := make(map[string]config.ProxySubscription, len(poolCfg.Subscriptions))
for name, sc := range poolCfg.Subscriptions {
if sc != nil {
subsCfg[name] = *sc
}
}
m.cfg.RUnlock()
m.mu.Lock()
endpoints := make([]*endpoint, 0, len(m.endpoints))
for _, ep := range m.endpoints {
endpoints = append(endpoints, ep)
}
sticky := make(map[string]string, len(m.sticky))
for k, v := range m.sticky {
sticky[k] = v
}
subs := make(map[string]*subState, len(m.subs))
for name, st := range m.subs {
cp := *st
subs[name] = &cp
}
lastErr := m.lastErr
m.mu.Unlock()
st := Status{
Enabled: poolCfg.Enabled,
CheckIntervalS: poolCfg.CheckIntervalSeconds,
CheckTimeoutS: poolCfg.CheckTimeoutSeconds,
CheckURL: poolCfg.CheckURL,
DefaultScheme: poolCfg.DefaultScheme,
XrayPath: poolCfg.XrayPath,
XrayVersion: poolCfg.XrayVersion,
XrayAutoDownload: poolCfg.XrayAutoDownload,
Xray: m.xray.status(),
Sticky: sticky,
LastError: lastErr,
}
for _, ep := range endpoints {
es := EntryStatus{
Name: ep.name, Kind: ep.kind, Scheme: ep.scheme, Target: ep.display,
Source: ep.source, Enabled: ep.enabled, Healthy: ep.healthy,
Checked: ep.checked, LatencyMs: ep.latencyMs, LastErr: ep.lastErr,
ParseError: ep.parseErr,
}
if !ep.checkedAt.IsZero() {
es.CheckedAt = ep.checkedAt.Unix()
}
st.Entries = append(st.Entries, es)
}
sort.Slice(st.Entries, func(i, j int) bool {
a, b := st.Entries[i], st.Entries[j]
if (a.Source == "") != (b.Source == "") {
return a.Source == ""
}
if a.Source != b.Source {
return a.Source < b.Source
}
return a.Name < b.Name
})
for name, sc := range subsCfg {
ss := SubStatus{Name: name, URL: sc.URL, Enabled: sc.Enabled, IntervalMinutes: sc.IntervalMinutes}
if stt := subs[name]; stt != nil {
ss.Nodes = len(stt.nodes)
ss.LastErr = stt.lastErr
if !stt.fetchedAt.IsZero() {
ss.FetchedAt = stt.fetchedAt.Unix()
}
}
st.Subscriptions = append(st.Subscriptions, ss)
}
sort.Slice(st.Subscriptions, func(i, j int) bool { return st.Subscriptions[i].Name < st.Subscriptions[j].Name })
return st
}
func newEndpoint(name, link, defaultScheme, source string) *endpoint {
ep := &endpoint{name: name, source: source, link: strings.TrimSpace(link)}
node, err := ParseNode(ep.link, defaultScheme)
if err != nil {
ep.kind = "invalid"
ep.parseErr = err.Error()
return ep
}
ep.kind = node.Kind
ep.scheme = node.Scheme
ep.display = node.Display
ep.outbound = node.Outbound
ep.url = node.URL
return ep
}
func (m *Manager) setErr(err error) {
m.mu.Lock()
defer m.mu.Unlock()
if err == nil {
m.lastErr = ""
return
}
m.lastErr = err.Error()
}
func shortHash(s string) string {
sum := sha256.Sum256([]byte(s))
return hex.EncodeToString(sum[:4])
}
func trimErr(err error) string {
s := err.Error()
if len(s) > 160 {
return s[:160] + "..."
}
return s
}
+76
View File
@@ -0,0 +1,76 @@
package proxypool
import (
"testing"
"github.com/nyoungo/dsfree2api/internal/config"
)
const examplePath = "../../config.example.toml"
func TestCandidatesStickySwitchOnFailure(t *testing.T) {
cfg, err := config.Load(examplePath)
if err != nil {
t.Fatalf("load config: %v", err)
}
cfg.ProxyPool.Enabled = true
cfg.ProxyPool.Entries = map[string]*config.ProxyEntry{
"a": {Link: "http://127.0.0.1:9", Enabled: true},
"b": {Link: "socks5://127.0.0.1:10", Enabled: true},
}
cfg.Sites["de"].Proxies = []string{"a", "b"}
m := New(cfg, t.TempDir(), nil)
m.Reload()
cands := m.Candidates("de")
if len(cands) != 2 || cands[0].Name != "a" || cands[0].Proxy != "http://127.0.0.1:9" {
t.Fatalf("candidates = %+v", cands)
}
// Both healthy: the sticky pick stays a.
m.mu.Lock()
for _, ep := range m.endpoints {
ep.checked = true
ep.healthy = true
}
m.mu.Unlock()
cands = m.Candidates("de")
if cands[0].Name != "a" {
t.Fatalf("sticky pick changed without failure: %+v", cands)
}
// a fails: the request path must immediately switch to b.
m.Report("http://127.0.0.1:9", false)
cands = m.Candidates("de")
if len(cands) != 1 || cands[0].Name != "b" {
t.Fatalf("after failure = %+v", cands)
}
// Everything unhealthy: fall back to the bound order so requests still try.
m.Report("socks5://127.0.0.1:10", false)
cands = m.Candidates("de")
if len(cands) != 2 || cands[0].Name != "a" {
t.Fatalf("all-unhealthy fallback = %+v", cands)
}
}
func TestCandidatesDisabledOrUnbound(t *testing.T) {
cfg, err := config.Load(examplePath)
if err != nil {
t.Fatalf("load config: %v", err)
}
cfg.ProxyPool.Enabled = false
cfg.Sites["de"].Proxies = []string{"a"}
m := New(cfg, t.TempDir(), nil)
m.Reload()
if got := m.Candidates("de"); got != nil {
t.Fatalf("disabled pool returned %+v", got)
}
cfg.ProxyPool.Enabled = true
m.Reload()
if got := m.Candidates("de"); got != nil {
t.Fatalf("unbound/unknown entry returned %+v", got)
}
}
+517
View File
@@ -0,0 +1,517 @@
// Package proxypool manages the egress endpoints used to reach the upstream
// sites: Xray share links driven by a managed Xray core, plus plain
// http/https/socks5 endpoints. Selection is sticky per site and health is
// probed in the background.
package proxypool
import (
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"net"
"net/url"
"strconv"
"strings"
)
// Node is a parsed proxy node.
type Node struct {
Kind string // "xray" (share link) or "url" (plain endpoint)
Scheme string // vless / vmess / trojan / ss / http / https / socks5
Display string // host:port, for the console
Link string // original input line
URL string // Kind == "url": ready-to-use proxy URL
Outbound map[string]any // Kind == "xray": Xray outbound object (tag added later)
}
// ParseNode parses one line: an Xray share link (vless/vmess/trojan/ss) or a
// plain endpoint. Accepted plain forms:
//
// scheme://user:pass@host:port
// user:pass@host:port (scheme = defaultScheme)
// host:port (scheme = defaultScheme)
// host:port:user:pass (scheme = defaultScheme)
func ParseNode(line, defaultScheme string) (*Node, error) {
line = strings.TrimSpace(line)
if line == "" {
return nil, errors.New("empty node")
}
switch {
case strings.HasPrefix(line, "vless://"):
return parseVless(line)
case strings.HasPrefix(line, "vmess://"):
return parseVmess(line)
case strings.HasPrefix(line, "trojan://"):
return parseTrojan(line)
case strings.HasPrefix(line, "ss://"):
return parseShadowsocks(line)
default:
return parsePlainEndpoint(line, defaultScheme)
}
}
func parseVless(raw string) (*Node, error) {
u, err := url.Parse(raw)
if err != nil {
return nil, fmt.Errorf("vless: %w", err)
}
id := u.User.Username()
host, port := u.Hostname(), u.Port()
if id == "" || host == "" || port == "" {
return nil, errors.New("vless: missing uuid / host / port")
}
q := u.Query()
user := map[string]any{"id": id, "encryption": "none"}
if flow := q.Get("flow"); flow != "" {
user["flow"] = flow
}
out := map[string]any{
"protocol": "vless",
"settings": map[string]any{
"vnext": []any{map[string]any{
"address": host, "port": portInt(port), "users": []any{user},
}},
},
"streamSettings": streamSettings(firstNonEmptyStr(q.Get("type"), "tcp"), q.Get("security"), q),
}
return &Node{Kind: "xray", Scheme: "vless", Display: net.JoinHostPort(host, port), Link: raw, Outbound: out}, nil
}
func parseVmess(raw string) (*Node, error) {
body := strings.TrimPrefix(raw, "vmess://")
if i := strings.IndexAny(body, "#"); i >= 0 {
body = body[:i]
}
dec, err := b64Decode(body)
if err != nil {
return nil, fmt.Errorf("vmess: invalid base64 payload: %w", err)
}
var v struct {
Add string `json:"add"`
Port any `json:"port"`
ID string `json:"id"`
Aid any `json:"aid"`
Scy string `json:"scy"`
Net string `json:"net"`
Type string `json:"type"`
Host string `json:"host"`
Path string `json:"path"`
TLS string `json:"tls"`
SNI string `json:"sni"`
FP string `json:"fp"`
ALPN string `json:"alpn"`
}
if err := json.Unmarshal([]byte(dec), &v); err != nil {
return nil, fmt.Errorf("vmess: bad json: %w", err)
}
port := toInt(v.Port)
if v.Add == "" || port == 0 || v.ID == "" {
return nil, errors.New("vmess: missing add / port / id")
}
security := strings.TrimSpace(v.Scy)
if security == "" {
security = "auto"
}
out := map[string]any{
"protocol": "vmess",
"settings": map[string]any{
"vnext": []any{map[string]any{
"address": v.Add, "port": port,
"users": []any{map[string]any{"id": v.ID, "alterId": toInt(v.Aid), "security": security}},
}},
},
}
network := strings.ToLower(strings.TrimSpace(v.Net))
if network == "" {
network = "tcp"
}
ss := map[string]any{"network": network}
switch network {
case "ws":
ws := map[string]any{}
if v.Path != "" {
ws["path"] = v.Path
}
if v.Host != "" {
ws["headers"] = map[string]any{"Host": v.Host}
}
ss["wsSettings"] = ws
case "h2":
h2 := map[string]any{}
if v.Path != "" {
h2["path"] = v.Path
}
if v.Host != "" {
h2["host"] = []any{v.Host}
}
ss["h2Settings"] = h2
case "grpc":
g := map[string]any{}
if v.Path != "" {
g["serviceName"] = v.Path
}
ss["grpcSettings"] = g
case "tcp":
if strings.EqualFold(v.Type, "http") {
req := map[string]any{"path": []any{firstNonEmptyStr(v.Path, "/")}}
if v.Host != "" {
req["headers"] = map[string]any{"Host": []any{v.Host}}
}
ss["tcpSettings"] = map[string]any{"header": map[string]any{"type": "http", "request": req}}
}
case "httpupgrade", "xhttp":
sub := map[string]any{}
if v.Path != "" {
sub["path"] = v.Path
}
if v.Host != "" {
sub["host"] = v.Host
}
ss[network+"Settings"] = sub
}
if strings.EqualFold(v.TLS, "tls") {
ss["security"] = "tls"
t := map[string]any{}
if sni := firstNonEmptyStr(v.SNI, v.Host); sni != "" {
t["serverName"] = sni
}
if v.FP != "" {
t["fingerprint"] = v.FP
}
if v.ALPN != "" {
t["alpn"] = strings.Split(v.ALPN, ",")
}
ss["tlsSettings"] = t
}
out["streamSettings"] = ss
return &Node{Kind: "xray", Scheme: "vmess", Display: net.JoinHostPort(v.Add, strconv.Itoa(port)), Link: raw, Outbound: out}, nil
}
func parseTrojan(raw string) (*Node, error) {
u, err := url.Parse(raw)
if err != nil {
return nil, fmt.Errorf("trojan: %w", err)
}
pass := u.User.Username()
host, port := u.Hostname(), u.Port()
if pass == "" || host == "" || port == "" {
return nil, errors.New("trojan: missing password / host / port")
}
q := u.Query()
security := q.Get("security")
if security == "" {
security = "tls"
}
out := map[string]any{
"protocol": "trojan",
"settings": map[string]any{
"servers": []any{map[string]any{"address": host, "port": portInt(port), "password": pass}},
},
"streamSettings": streamSettings(firstNonEmptyStr(q.Get("type"), "tcp"), security, q),
}
return &Node{Kind: "xray", Scheme: "trojan", Display: net.JoinHostPort(host, port), Link: raw, Outbound: out}, nil
}
func parseShadowsocks(raw string) (*Node, error) {
body := strings.TrimPrefix(raw, "ss://")
if i := strings.Index(body, "#"); i >= 0 {
body = body[:i]
}
var method, pass, hostport string
if at := strings.LastIndex(body, "@"); at >= 0 {
userinfo, hp := body[:at], body[at+1:]
if i := strings.IndexAny(hp, "?"); i >= 0 {
hp = hp[:i]
}
if strings.Contains(userinfo, ":") {
// plain method:password (uncommon)
} else {
dec, err := b64Decode(userinfo)
if err != nil {
return nil, errors.New("ss: cannot decode credentials")
}
userinfo = dec
}
mp := strings.SplitN(userinfo, ":", 2)
if len(mp) != 2 || mp[0] == "" || mp[1] == "" {
return nil, errors.New("ss: bad credentials")
}
method, pass, hostport = mp[0], mp[1], hp
} else {
// legacy: ss://base64(method:password@host:port)
dec, err := b64Decode(body)
if err != nil {
return nil, errors.New("ss: invalid base64 payload")
}
at := strings.LastIndex(dec, "@")
if at < 0 {
return nil, errors.New("ss: missing @host")
}
mp := strings.SplitN(dec[:at], ":", 2)
if len(mp) != 2 {
return nil, errors.New("ss: bad credentials")
}
method, pass, hostport = mp[0], mp[1], dec[at+1:]
}
host, port, err := splitHostPort(hostport)
if err != nil {
return nil, fmt.Errorf("ss: %w", err)
}
out := map[string]any{
"protocol": "shadowsocks",
"settings": map[string]any{
"servers": []any{map[string]any{"address": host, "port": port, "method": method, "password": pass}},
},
}
return &Node{Kind: "xray", Scheme: "ss", Display: net.JoinHostPort(host, strconv.Itoa(port)), Link: raw, Outbound: out}, nil
}
func parsePlainEndpoint(line, defaultScheme string) (*Node, error) {
if defaultScheme = strings.ToLower(strings.TrimSpace(defaultScheme)); defaultScheme == "" {
defaultScheme = "http"
}
build := func(scheme, host, port, user, pass string) (*Node, error) {
scheme = normalizeScheme(scheme)
if scheme == "" {
return nil, fmt.Errorf("unsupported proxy scheme %q (http/https/socks5)", scheme)
}
if host == "" || port == "" {
return nil, errors.New("missing host or port")
}
u := &url.URL{Scheme: scheme, Host: net.JoinHostPort(host, port)}
if user != "" {
u.User = url.UserPassword(user, pass)
}
return &Node{
Kind: "url", Scheme: scheme,
Display: net.JoinHostPort(host, port),
Link: line, URL: u.String(),
}, nil
}
switch {
case strings.Contains(line, "://"):
u, err := url.Parse(line)
if err != nil {
return nil, fmt.Errorf("bad proxy url: %w", err)
}
user, pass := "", ""
if u.User != nil {
user = u.User.Username()
pass, _ = u.User.Password()
}
return build(u.Scheme, u.Hostname(), u.Port(), user, pass)
case strings.Contains(line, "@"):
at := strings.LastIndex(line, "@")
userinfo, hostport := line[:at], line[at+1:]
mp := strings.SplitN(userinfo, ":", 2)
user, pass := mp[0], ""
if len(mp) == 2 {
pass = mp[1]
}
host, port, err := splitHostPort(hostport)
if err != nil {
return nil, err
}
return build(defaultScheme, host, strconv.Itoa(port), user, pass)
default:
parts := strings.Split(line, ":")
switch len(parts) {
case 2:
return build(defaultScheme, parts[0], parts[1], "", "")
case 4:
// ip:port:user:pass
return build(defaultScheme, parts[0], parts[1], parts[2], parts[3])
default:
return nil, fmt.Errorf("unrecognized endpoint %q", line)
}
}
}
// ParseSubscriptionBody turns a subscription response into node lines. The
// body may be plain text (one node per line) or a base64 blob of that text.
func ParseSubscriptionBody(raw []byte) []string {
s := strings.TrimSpace(string(raw))
if s == "" {
return nil
}
if !strings.Contains(s, "://") {
if dec, err := b64Decode(strings.ReplaceAll(s, "\n", "")); err == nil {
s = dec
}
}
out := make([]string, 0, 16)
for _, line := range strings.Split(s, "\n") {
line = strings.TrimSpace(line)
if line == "" || strings.HasPrefix(line, "#") {
continue
}
out = append(out, line)
}
return out
}
// ── helpers ──────────────────────────────────────────────────────
// streamSettings builds Xray streamSettings from share-link query params.
func streamSettings(network, security string, q url.Values) map[string]any {
network = strings.ToLower(strings.TrimSpace(network))
if network == "" {
network = "tcp"
}
ss := map[string]any{"network": network}
switch network {
case "ws":
ws := map[string]any{}
if p := q.Get("path"); p != "" {
ws["path"] = p
}
if h := q.Get("host"); h != "" {
ws["headers"] = map[string]any{"Host": h}
}
ss["wsSettings"] = ws
case "grpc":
g := map[string]any{}
if s := q.Get("serviceName"); s != "" {
g["serviceName"] = s
}
ss["grpcSettings"] = g
case "httpupgrade":
h := map[string]any{}
if p := q.Get("path"); p != "" {
h["path"] = p
}
if hh := q.Get("host"); hh != "" {
h["host"] = hh
}
ss["httpupgradeSettings"] = h
case "xhttp":
x := map[string]any{}
if p := q.Get("path"); p != "" {
x["path"] = p
}
if hh := q.Get("host"); hh != "" {
x["host"] = hh
}
if m := q.Get("mode"); m != "" {
x["mode"] = m
}
ss["xhttpSettings"] = x
case "tcp":
if strings.EqualFold(q.Get("headerType"), "http") {
req := map[string]any{}
if h := q.Get("host"); h != "" {
req["headers"] = map[string]any{"Host": []any{h}}
}
if p := q.Get("path"); p != "" {
req["path"] = []any{p}
}
ss["tcpSettings"] = map[string]any{"header": map[string]any{"type": "http", "request": req}}
}
}
switch strings.ToLower(security) {
case "tls":
t := map[string]any{}
if sni := firstNonEmptyStr(q.Get("sni"), q.Get("host")); sni != "" {
t["serverName"] = sni
}
if fp := q.Get("fp"); fp != "" {
t["fingerprint"] = fp
}
if alpn := q.Get("alpn"); alpn != "" {
t["alpn"] = strings.Split(alpn, ",")
}
ss["security"] = "tls"
ss["tlsSettings"] = t
case "reality":
r := map[string]any{}
if sni := q.Get("sni"); sni != "" {
r["serverName"] = sni
}
if fp := q.Get("fp"); fp != "" {
r["fingerprint"] = fp
}
if pbk := q.Get("pbk"); pbk != "" {
r["publicKey"] = pbk
}
if sid := q.Get("sid"); sid != "" {
r["shortId"] = sid
}
if spx := q.Get("spx"); spx != "" {
r["spiderX"] = spx
}
ss["security"] = "reality"
ss["realitySettings"] = r
}
return ss
}
func normalizeScheme(s string) string {
switch strings.ToLower(strings.TrimSpace(s)) {
case "http":
return "http"
case "https":
return "https"
case "socks", "socks5", "socks5h":
return "socks5"
default:
return ""
}
}
func b64Decode(s string) (string, error) {
s = strings.TrimSpace(s)
trimmed := strings.TrimRight(s, "=")
for _, enc := range []*base64.Encoding{
base64.RawURLEncoding, base64.URLEncoding,
base64.RawStdEncoding, base64.StdEncoding,
} {
if b, err := enc.DecodeString(trimmed); err == nil {
return string(b), nil
}
}
return "", errors.New("invalid base64")
}
func firstNonEmptyStr(vals ...string) string {
for _, v := range vals {
if strings.TrimSpace(v) != "" {
return v
}
}
return ""
}
func portInt(s string) int {
n, _ := strconv.Atoi(strings.TrimSpace(s))
return n
}
func toInt(v any) int {
switch x := v.(type) {
case float64:
return int(x)
case int:
return x
case int64:
return int(x)
case string:
n, _ := strconv.Atoi(strings.TrimSpace(x))
return n
default:
return 0
}
}
func splitHostPort(hostport string) (string, int, error) {
host, port, err := net.SplitHostPort(strings.TrimSpace(hostport))
if err != nil {
return "", 0, fmt.Errorf("bad host:port %q", hostport)
}
n, err := strconv.Atoi(port)
if err != nil || n < 1 || n > 65535 {
return "", 0, fmt.Errorf("bad port in %q", hostport)
}
return host, n, nil
}
+145
View File
@@ -0,0 +1,145 @@
package proxypool
import (
"encoding/base64"
"testing"
)
func TestParseVless(t *testing.T) {
line := "vless://11111111-2222-3333-4444-555555555555@example.com:443" +
"?type=ws&security=tls&sni=sni.example.com&path=%2Fws&host=cdn.example.com&fp=chrome#node"
n, err := ParseNode(line, "http")
if err != nil {
t.Fatalf("parse: %v", err)
}
if n.Kind != "xray" || n.Scheme != "vless" || n.Display != "example.com:443" {
t.Fatalf("node = %+v", n)
}
if n.Outbound["protocol"] != "vless" {
t.Fatalf("protocol = %v", n.Outbound["protocol"])
}
settings := n.Outbound["settings"].(map[string]any)
vnext := settings["vnext"].([]any)[0].(map[string]any)
if vnext["address"] != "example.com" || vnext["port"] != 443 {
t.Fatalf("vnext = %+v", vnext)
}
users := vnext["users"].([]any)[0].(map[string]any)
if users["id"] != "11111111-2222-3333-4444-555555555555" || users["encryption"] != "none" {
t.Fatalf("users = %+v", users)
}
ss := n.Outbound["streamSettings"].(map[string]any)
if ss["network"] != "ws" || ss["security"] != "tls" {
t.Fatalf("stream = %+v", ss)
}
ws := ss["wsSettings"].(map[string]any)
if ws["path"] != "/ws" {
t.Fatalf("ws = %+v", ws)
}
tls := ss["tlsSettings"].(map[string]any)
if tls["serverName"] != "sni.example.com" {
t.Fatalf("tls = %+v", tls)
}
}
func TestParseVmess(t *testing.T) {
payload := `{"v":"2","ps":"n","add":"vm.example.com","port":"443","id":"uuid-1",` +
`"aid":"0","scy":"auto","net":"ws","type":"none","host":"vm.example.com",` +
`"path":"/vm","tls":"tls","sni":"vm.example.com"}`
line := "vmess://" + base64.StdEncoding.EncodeToString([]byte(payload))
n, err := ParseNode(line, "")
if err != nil {
t.Fatalf("parse: %v", err)
}
if n.Kind != "xray" || n.Scheme != "vmess" || n.Display != "vm.example.com:443" {
t.Fatalf("node = %+v", n)
}
ss := n.Outbound["streamSettings"].(map[string]any)
if ss["network"] != "ws" || ss["security"] != "tls" {
t.Fatalf("stream = %+v", ss)
}
if ss["wsSettings"].(map[string]any)["path"] != "/vm" {
t.Fatalf("ws = %+v", ss["wsSettings"])
}
}
func TestParseTrojan(t *testing.T) {
n, err := ParseNode("trojan://secret@tr.example.com:443?security=tls&sni=tr.example.com#t", "")
if err != nil {
t.Fatalf("parse: %v", err)
}
if n.Scheme != "trojan" || n.Display != "tr.example.com:443" {
t.Fatalf("node = %+v", n)
}
settings := n.Outbound["settings"].(map[string]any)
srv := settings["servers"].([]any)[0].(map[string]any)
if srv["address"] != "tr.example.com" || srv["port"] != 443 || srv["password"] != "secret" {
t.Fatalf("server = %+v", srv)
}
}
func TestParseShadowsocks(t *testing.T) {
// SIP002: base64(method:password)@host:port
ui := base64.RawURLEncoding.EncodeToString([]byte("aes-256-gcm:pass:word"))
n, err := ParseNode("ss://"+ui+"@ss.example.com:8388#x", "")
if err != nil {
t.Fatalf("sip002 parse: %v", err)
}
srv := n.Outbound["settings"].(map[string]any)["servers"].([]any)[0].(map[string]any)
if srv["method"] != "aes-256-gcm" || srv["password"] != "pass:word" || srv["address"] != "ss.example.com" {
t.Fatalf("server = %+v", srv)
}
// legacy: base64(method:password@host:port)
legacy := base64.RawURLEncoding.EncodeToString([]byte("aes-256-gcm:secret@ss.example.com:8388"))
n2, err := ParseNode("ss://"+legacy, "")
if err != nil {
t.Fatalf("legacy parse: %v", err)
}
srv2 := n2.Outbound["settings"].(map[string]any)["servers"].([]any)[0].(map[string]any)
if srv2["password"] != "secret" || srv2["address"] != "ss.example.com" || srv2["port"] != 8388 {
t.Fatalf("legacy server = %+v", srv2)
}
}
func TestParsePlainEndpoints(t *testing.T) {
cases := []struct {
line, def, wantURL, wantDisplay string
}{
{"socks5://u:p@1.2.3.4:1080", "http", "socks5://u:p@1.2.3.4:1080", "1.2.3.4:1080"},
{"http://1.2.3.4:8080", "socks5", "http://1.2.3.4:8080", "1.2.3.4:8080"},
{"1.2.3.4:8080", "http", "http://1.2.3.4:8080", "1.2.3.4:8080"},
{"1.2.3.4:8080", "socks5", "socks5://1.2.3.4:8080", "1.2.3.4:8080"},
{"u:p@1.2.3.4:1080", "http", "http://u:p@1.2.3.4:1080", "1.2.3.4:1080"},
{"1.2.3.4:8080:u:p", "http", "http://u:p@1.2.3.4:8080", "1.2.3.4:8080"},
}
for _, c := range cases {
n, err := ParseNode(c.line, c.def)
if err != nil {
t.Fatalf("%q: %v", c.line, err)
}
if n.Kind != "url" || n.URL != c.wantURL || n.Display != c.wantDisplay {
t.Errorf("%q => kind=%s url=%q display=%q", c.line, n.Kind, n.URL, n.Display)
}
}
}
func TestParseSubscriptionBody(t *testing.T) {
plain := "vless://a@b:1\n# comment\nsocks5://x:y@1.2.3.4:5\n\n"
got := ParseSubscriptionBody([]byte(plain))
if len(got) != 2 || got[0] != "vless://a@b:1" || got[1] != "socks5://x:y@1.2.3.4:5" {
t.Fatalf("plain parse = %v", got)
}
enc := base64.StdEncoding.EncodeToString([]byte(plain))
got2 := ParseSubscriptionBody([]byte(enc))
if len(got2) != 2 || got2[0] != got[0] {
t.Fatalf("base64 parse = %v", got2)
}
}
func TestParseNodeRejectsJunk(t *testing.T) {
for _, bad := range []string{"", "hello world", "vless://no-port", "ss://@@@", "1.2.3.4", "ftp://1.2.3.4:21"} {
if _, err := ParseNode(bad, "http"); err == nil {
t.Errorf("%q should fail to parse", bad)
}
}
}
+498
View File
@@ -0,0 +1,498 @@
package proxypool
import (
"archive/zip"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"log/slog"
"net"
"net/http"
"os"
"os/exec"
"path/filepath"
"runtime"
"strings"
"sync"
"time"
)
// xrayNode is one Xray-backed endpoint, ready to be written into the core
// config.
type xrayNode struct {
Name string
Outbound map[string]any
}
// xrayCore owns the managed Xray process: one socks inbound per pool node,
// each routed to its own outbound.
type xrayCore struct {
dir string
log *slog.Logger
mu sync.Mutex
path string
proc *exec.Cmd
exited chan struct{}
running bool
pid int
cfgPath string
cfgHash string
ports map[string]int
lastErr string
tail *logTail
}
// XrayStatus is the console view of the core.
type XrayStatus struct {
Path string `json:"path"`
Running bool `json:"running"`
PID int `json:"pid"`
Error string `json:"error,omitempty"`
}
func newXrayCore(dir string, log *slog.Logger) *xrayCore {
if log == nil {
log = slog.Default()
}
return &xrayCore{dir: dir, log: log, ports: map[string]int{}}
}
func (c *xrayCore) status() XrayStatus {
c.mu.Lock()
defer c.mu.Unlock()
return XrayStatus{Path: c.path, Running: c.running, PID: c.pid, Error: c.lastErr}
}
// ensure resolves the Xray binary: explicit path, then a previously downloaded
// copy, then (optionally) a fresh download. The download runs unlocked so
// status reads never stall behind a slow network.
func (c *xrayCore) ensure(explicitPath string, autoDownload bool, version string) error {
c.mu.Lock()
if c.path != "" {
if _, err := os.Stat(c.path); err == nil {
c.mu.Unlock()
return nil
}
c.path = ""
}
if p := strings.TrimSpace(explicitPath); p != "" {
if _, err := os.Stat(p); err != nil {
c.mu.Unlock()
return fmt.Errorf("xray_path: %w", err)
}
c.path = p
c.mu.Unlock()
return nil
}
target := filepath.Join(c.dir, "xray"+exeSuffix())
if _, err := os.Stat(target); err == nil {
c.path = target
c.mu.Unlock()
return nil
}
if !autoDownload {
c.mu.Unlock()
return errors.New("xray binary not found — set [proxypool].xray_path, or enable xray_auto_download")
}
c.mu.Unlock()
if err := c.download(target, version); err != nil {
c.mu.Lock()
c.lastErr = err.Error()
c.mu.Unlock()
return err
}
c.mu.Lock()
c.path = target
c.mu.Unlock()
return nil
}
func (c *xrayCore) download(target, version string) error {
if err := os.MkdirAll(c.dir, 0o755); err != nil {
return err
}
osName := map[string]string{"windows": "windows", "linux": "linux", "darwin": "macos"}[runtime.GOOS]
arch := map[string]string{"amd64": "64", "arm64": "arm64-v8a", "386": "32", "arm": "arm32-v7a"}[runtime.GOARCH]
if osName == "" || arch == "" {
return fmt.Errorf("xray: unsupported platform %s/%s", runtime.GOOS, runtime.GOARCH)
}
asset := fmt.Sprintf("Xray-%s-%s.zip", osName, arch)
downloadURL := "https://github.com/XTLS/Xray-core/releases/latest/download/" + asset
if v := strings.TrimSpace(version); v != "" {
if !strings.HasPrefix(v, "v") {
v = "v" + v
}
downloadURL = fmt.Sprintf("https://github.com/XTLS/Xray-core/releases/download/%s/%s", v, asset)
}
c.log.Info("downloading xray core", "asset", asset)
zipPath := target + ".download"
if err := httpDownload(downloadURL, zipPath, 180*time.Second); err != nil {
return err
}
defer os.Remove(zipPath)
if want, err := fetchDigest(downloadURL + ".dgst"); err != nil {
c.log.Warn("xray checksum unavailable, skipping verification", "error", err)
} else if want != "" {
got, err := fileSHA256(zipPath)
if err != nil {
return err
}
if !strings.EqualFold(got, want) {
return fmt.Errorf("xray checksum mismatch: got %s want %s", got, want)
}
}
if err := extractZipBinary(zipPath, target); err != nil {
return err
}
_ = os.Chmod(target, 0o755)
c.log.Info("xray core ready", "path", target)
return nil
}
// configure regenerates the core config for the given nodes and restarts the
// process when something changed. Ports are kept stable across restarts so
// cached cookies stay valid.
func (c *xrayCore) configure(nodes []xrayNode) error {
c.mu.Lock()
defer c.mu.Unlock()
if len(nodes) == 0 {
c.stopLocked()
c.cfgHash = ""
return nil
}
if c.path == "" {
return errors.New("xray binary unavailable")
}
for _, n := range nodes {
if c.ports[n.Name] == 0 {
p, err := freePort()
if err != nil {
return err
}
c.ports[n.Name] = p
}
}
if c.configHash(nodes) == c.cfgHash && c.running {
return nil
}
// A restart is due. Stop first so our own listeners do not count as
// "port busy", then reclaim or reassign every assigned port — another
// local core may have raced us for one, since the OS only guaranteed it
// at allocation time.
c.stopLocked()
for _, n := range nodes {
if !portAvailable(c.ports[n.Name]) {
p, err := freePort()
if err != nil {
return err
}
c.ports[n.Name] = p
}
}
raw, err := json.Marshal(buildXrayConfig(nodes, c.ports))
if err != nil {
return err
}
sum := sha256.Sum256(raw)
c.cfgHash = hex.EncodeToString(sum[:])
cfgPath := filepath.Join(c.dir, "config.json")
if err := os.MkdirAll(c.dir, 0o755); err != nil {
return err
}
if err := os.WriteFile(cfgPath, raw, 0o600); err != nil {
return err
}
c.cfgPath = cfgPath
return c.startLocked(nodes)
}
// configHash is the digest of the config that would be generated for nodes
// with the current port assignments.
func (c *xrayCore) configHash(nodes []xrayNode) string {
raw, err := json.Marshal(buildXrayConfig(nodes, c.ports))
if err != nil {
return ""
}
sum := sha256.Sum256(raw)
return hex.EncodeToString(sum[:])
}
// portAvailable reports whether 127.0.0.1:port can still be claimed.
func portAvailable(port int) bool {
if port <= 0 {
return false
}
l, err := net.Listen("tcp", fmt.Sprintf("127.0.0.1:%d", port))
if err != nil {
return false
}
l.Close()
return true
}
func (c *xrayCore) startLocked(nodes []xrayNode) error {
cmd := exec.Command(c.path, "run", "-c", c.cfgPath)
c.tail = &logTail{max: 40}
cmd.Stdout = c.tail
cmd.Stderr = c.tail
if err := cmd.Start(); err != nil {
c.lastErr = "start xray: " + err.Error()
return errors.New(c.lastErr)
}
c.proc = cmd
c.pid = cmd.Process.Pid
c.running = true
c.lastErr = ""
exited := make(chan struct{})
c.exited = exited
go func() {
_ = cmd.Wait()
c.mu.Lock()
if c.proc == cmd {
c.running = false
c.pid = 0
if t := c.tail.Last(); t != "" {
c.lastErr = "xray exited: " + t
} else {
c.lastErr = "xray exited"
}
}
c.mu.Unlock()
close(exited)
}()
// Wait until the local socks inbounds answer.
deadline := time.Now().Add(8 * time.Second)
for time.Now().Before(deadline) {
ok := true
for _, n := range nodes {
conn, err := net.DialTimeout("tcp", fmt.Sprintf("127.0.0.1:%d", c.ports[n.Name]), 300*time.Millisecond)
if err != nil {
ok = false
break
}
conn.Close()
}
if ok {
return nil
}
if !c.running {
break
}
time.Sleep(200 * time.Millisecond)
}
if !c.running {
if c.lastErr == "" {
c.lastErr = "xray failed to start"
}
return errors.New(c.lastErr)
}
c.lastErr = "xray socks ports did not come up in time"
return errors.New(c.lastErr)
}
func (c *xrayCore) stopLocked() {
if c.proc != nil && c.proc.Process != nil {
_ = c.proc.Process.Kill()
}
if c.exited != nil {
select {
case <-c.exited:
case <-time.After(3 * time.Second):
}
}
c.proc = nil
c.exited = nil
c.running = false
c.pid = 0
}
func (c *xrayCore) stop() {
c.mu.Lock()
defer c.mu.Unlock()
c.stopLocked()
}
// port returns the local socks port assigned to a node name.
func (c *xrayCore) port(name string) int {
c.mu.Lock()
defer c.mu.Unlock()
return c.ports[name]
}
func buildXrayConfig(nodes []xrayNode, ports map[string]int) map[string]any {
inbounds := make([]any, 0, len(nodes))
outbounds := make([]any, 0, len(nodes)+2)
rules := make([]any, 0, len(nodes))
for i, n := range nodes {
inTag := fmt.Sprintf("in-%d", i)
outTag := fmt.Sprintf("out-%d", i)
inbounds = append(inbounds, map[string]any{
"tag": inTag, "listen": "127.0.0.1", "port": ports[n.Name],
"protocol": "socks",
"settings": map[string]any{"udp": false, "auth": "noauth"},
})
ob := make(map[string]any, len(n.Outbound)+1)
for k, v := range n.Outbound {
ob[k] = v
}
ob["tag"] = outTag
outbounds = append(outbounds, ob)
rules = append(rules, map[string]any{"type": "field", "inboundTag": []any{inTag}, "outboundTag": outTag})
}
outbounds = append(outbounds,
map[string]any{"tag": "direct", "protocol": "freedom"},
map[string]any{"tag": "blocked", "protocol": "blackhole"},
)
return map[string]any{
"log": map[string]any{"loglevel": "warning", "access": "none"},
"inbounds": inbounds,
"outbounds": outbounds,
"routing": map[string]any{"domainStrategy": "AsIs", "rules": rules},
}
}
// ── platform / io helpers ────────────────────────────────────────
func exeSuffix() string {
if runtime.GOOS == "windows" {
return ".exe"
}
return ""
}
func freePort() (int, error) {
l, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
return 0, err
}
defer l.Close()
return l.Addr().(*net.TCPAddr).Port, nil
}
func httpDownload(url, dst string, timeout time.Duration) error {
client := &http.Client{Timeout: timeout}
resp, err := client.Get(url)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("download %s: http %d", url, resp.StatusCode)
}
f, err := os.Create(dst)
if err != nil {
return err
}
defer f.Close()
if _, err := io.Copy(f, io.LimitReader(resp.Body, 256<<20)); err != nil {
return err
}
return f.Sync()
}
func fetchDigest(url string) (string, error) {
client := &http.Client{Timeout: 20 * time.Second}
resp, err := client.Get(url)
if err != nil {
return "", err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return "", fmt.Errorf("http %d", resp.StatusCode)
}
raw, err := io.ReadAll(io.LimitReader(resp.Body, 64<<10))
if err != nil {
return "", err
}
for _, line := range strings.Split(string(raw), "\n") {
key, val, ok := strings.Cut(line, ":")
if !ok {
continue
}
if strings.Contains(strings.ToUpper(key), "SHA2-256") || strings.Contains(strings.ToUpper(key), "SHA256") {
if v := strings.TrimSpace(strings.Fields(val)[0]); v != "" {
return v, nil
}
}
}
return "", errors.New("no sha256 digest found")
}
func fileSHA256(path string) (string, error) {
f, err := os.Open(path)
if err != nil {
return "", err
}
defer f.Close()
h := sha256.New()
if _, err := io.Copy(h, f); err != nil {
return "", err
}
return hex.EncodeToString(h.Sum(nil)), nil
}
func extractZipBinary(zipPath, target string) error {
r, err := zip.OpenReader(zipPath)
if err != nil {
return fmt.Errorf("open archive: %w", err)
}
defer r.Close()
want := "xray" + exeSuffix()
for _, f := range r.File {
if !strings.EqualFold(filepath.Base(f.Name), want) {
continue
}
rc, err := f.Open()
if err != nil {
return err
}
defer rc.Close()
out, err := os.OpenFile(target, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, 0o755)
if err != nil {
return err
}
if _, err := io.Copy(out, rc); err != nil {
out.Close()
return err
}
return out.Close()
}
return errors.New("no xray binary inside the archive")
}
// logTail keeps the last few process log lines for status reporting.
type logTail struct {
mu sync.Mutex
lines []string
max int
}
func (t *logTail) Write(p []byte) (int, error) {
t.mu.Lock()
defer t.mu.Unlock()
for _, line := range strings.Split(strings.TrimRight(string(p), "\r\n"), "\n") {
line = strings.TrimSpace(line)
if line == "" {
continue
}
t.lines = append(t.lines, line)
}
if len(t.lines) > t.max {
t.lines = t.lines[len(t.lines)-t.max:]
}
return len(p), nil
}
func (t *logTail) Last() string {
t.mu.Lock()
defer t.mu.Unlock()
if len(t.lines) == 0 {
return ""
}
return t.lines[len(t.lines)-1]
}
+80
View File
@@ -0,0 +1,80 @@
package proxypool
import (
"net"
"os"
"strings"
"testing"
"github.com/nyoungo/dsfree2api/internal/config"
)
// TestXrayCoreSmoke drives the managed core against the locally placed
// binary (data/xray.exe) with a syntactically real VLESS node. It never
// connects upstream — it only proves that ensure() picks up the local
// binary, configure() starts the process and the local socks inbound is
// live. Skipped unless opted in:
//
// $env:DSFREE_XRAY_SMOKE=1; go test ./internal/proxypool -run Smoke -v
func TestXrayCoreSmoke(t *testing.T) {
if os.Getenv("DSFREE_XRAY_SMOKE") == "" {
t.Skip("set DSFREE_XRAY_SMOKE=1 to run the local xray smoke test")
}
cfg, err := config.Load(examplePath)
if err != nil {
t.Fatalf("load config: %v", err)
}
cfg.ProxyPool.Enabled = true
cfg.ProxyPool.XrayAutoDownload = false // the local binary must be used as-is
cfg.ProxyPool.Entries = map[string]*config.ProxyEntry{
"smoke": {
Link: "vless://11111111-2222-3333-4444-555555555555@example.com:443" +
"?security=tls&type=tcp&sni=example.com",
Enabled: true,
},
}
cfg.Sites["de"].Proxies = []string{"smoke"}
m := New(cfg, "../../data", nil)
m.Reload()
st := m.Status()
if !st.Xray.Running {
t.Fatalf("xray core did not start: %+v", st.Xray)
}
t.Logf("xray running: pid=%d path=%s", st.Xray.PID, st.Xray.Path)
cands := m.Candidates("de")
if len(cands) != 1 || !strings.HasPrefix(cands[0].Proxy, "socks5://127.0.0.1:") {
t.Fatalf("candidates = %+v", cands)
}
t.Logf("candidate proxy: %s", cands[0].Proxy)
m.xray.stop()
if m.Status().Xray.Running {
t.Fatal("xray still running after stop")
}
// Port stability: a restart re-claims the same socks port, so cached
// cookies (keyed by the local proxy URL) stay valid.
m.Reload()
again := m.Candidates("de")
if len(again) != 1 || again[0].Proxy != cands[0].Proxy {
t.Fatalf("port not stable across restart: %q -> %+v", cands[0].Proxy, again)
}
m.xray.stop()
// Race self-heal: steal the assigned port; the next start must move away.
port := strings.TrimPrefix(cands[0].Proxy, "socks5://127.0.0.1:")
ln, err := net.Listen("tcp", "127.0.0.1:"+port)
if err != nil {
t.Fatalf("occupy port %s: %v", port, err)
}
defer ln.Close()
m.Reload()
third := m.Candidates("de")
if len(third) != 1 || third[0].Proxy == cands[0].Proxy {
t.Fatalf("stolen port was not reassigned: %+v", third)
}
t.Logf("port moved %s -> %s", cands[0].Proxy, third[0].Proxy)
m.xray.stop()
}
+357
View File
@@ -0,0 +1,357 @@
// Package quota watches the upstream sites' guest-token balances so the
// console and logs show how much free daily quota is left before a site cuts
// requests off. Readings reuse the pooled cookies and egress of the chat
// path, so they describe the identity requests are actually debited from.
package quota
import (
"context"
"encoding/json"
"errors"
"fmt"
"log/slog"
"net/http"
"sort"
"strconv"
"strings"
"sync"
"time"
"github.com/nyoungo/dsfree2api/internal/config"
"github.com/nyoungo/dsfree2api/internal/httpx"
"github.com/nyoungo/dsfree2api/internal/proxypool"
"github.com/nyoungo/dsfree2api/internal/turnstile"
)
// Balance is one site × bot quota reading. Empty Error means the reading was
// taken successfully; Limit/Used/Remaining come from the site's own balance
// endpoint (the guest daily free allowance).
type Balance struct {
Site string `json:"site"`
BotID int `json:"bot_id"`
Model string `json:"model,omitempty"`
Route string `json:"route"`
PaidBalance int64 `json:"paid_balance"`
Limit int64 `json:"limit"`
Used int64 `json:"used"`
Remaining int64 `json:"remaining"`
Ratio float64 `json:"ratio"`
ResetPeriod string `json:"reset_period,omitempty"`
CheckedAt int64 `json:"checked_at"`
Error string `json:"error,omitempty"`
}
// Snapshot is the console view of the sentinel.
type Snapshot struct {
Enabled bool `json:"enabled"`
Running bool `json:"running"`
CheckS int `json:"check_seconds"`
WarnRatio float64 `json:"warn_ratio"`
Entries []Balance `json:"entries"`
}
type botRef struct {
id int
model string
}
// Watcher polls each site's balance REST endpoint on an interval. One slot
// per site × bot; slots without verified cookies are reported as such and
// never trigger solves by themselves.
type Watcher struct {
cfg *config.Config
log *slog.Logger
ts *turnstile.Solver
pool *proxypool.Manager
// newSession is a test seam; nil uses the real Chrome-impersonating one.
newSession func(proxy string) (httpx.Session, error)
mu sync.Mutex
entries map[string]*Balance // key: site|bot
warned map[string]bool
lastErr map[string]string
running bool
}
// New builds a watcher. Start launches the loop when [quota].enabled.
func New(cfg *config.Config, log *slog.Logger, ts *turnstile.Solver, pool *proxypool.Manager) *Watcher {
if log == nil {
log = slog.Default()
}
return &Watcher{
cfg: cfg,
log: log,
ts: ts,
pool: pool,
entries: map[string]*Balance{},
warned: map[string]bool{},
lastErr: map[string]string{},
}
}
// Start launches the polling loop; no-op unless [quota].enabled.
func (w *Watcher) Start(ctx context.Context) {
w.cfg.RLock()
enabled := w.cfg.Quota.Enabled
w.cfg.RUnlock()
if !enabled || w.ts == nil {
return
}
go w.loop(ctx)
}
func (w *Watcher) loop(ctx context.Context) {
w.mu.Lock()
w.running = true
w.mu.Unlock()
defer func() {
w.mu.Lock()
w.running = false
w.mu.Unlock()
}()
// First sweep after a short settle delay (pool/cookies may still be
// warming up), then on every check interval.
timer := time.NewTimer(20 * time.Second)
defer timer.Stop()
for {
select {
case <-ctx.Done():
return
case <-timer.C:
w.Sweep(ctx)
w.cfg.RLock()
iv := time.Duration(w.cfg.Quota.CheckSecondsValue()) * time.Second
w.cfg.RUnlock()
timer.Reset(iv)
}
}
}
// Sweep queries every enabled site once. Exposed for tests and manual runs.
func (w *Watcher) Sweep(ctx context.Context) {
if w.ts == nil {
return
}
w.cfg.RLock()
sites := map[string]config.Site{}
for code, st := range w.cfg.Sites {
if st != nil && st.Enabled {
sites[code] = *st
}
}
bots := map[string][]botRef{}
for id, m := range w.cfg.Models {
if m == nil || !m.Enabled || m.BotID <= 0 {
continue
}
bots[m.Site] = append(bots[m.Site], botRef{id: m.BotID, model: id})
}
warnRatio := w.cfg.Quota.WarnRatioValue()
w.cfg.RUnlock()
codes := make([]string, 0, len(sites))
for code := range sites {
codes = append(codes, code)
}
sort.Strings(codes)
for _, code := range codes {
refs := bots[code]
if len(refs) == 0 {
continue
}
sort.Slice(refs, func(i, j int) bool { return refs[i].id < refs[j].id })
w.checkSite(ctx, sites[code], refs, warnRatio)
}
}
func (w *Watcher) checkSite(ctx context.Context, site config.Site, refs []botRef, warnRatio float64) {
route := w.routeFor(site.Code)
if !w.ts.CookiesReady(site.Code, route) {
w.markAll(site.Code, refs, route, errors.New("no verified cookies for this route yet"))
return
}
sess, err := w.session(route)
if err != nil {
w.markAll(site.Code, refs, route, err)
return
}
defer sess.Close()
if err := w.ts.ApplyValidCookies(ctx, sess, &site, route); err != nil {
w.markAll(site.Code, refs, route, err)
return
}
// Guests need no REST nonce: the visitor id travels as the dsgt_gid
// cookie and the balance endpoint keys on it.
if gid := sess.Cookies()["dsgt_gid"]; gid == "" {
w.markAll(site.Code, refs, route, errors.New("no dsgt_gid cookie for this route yet"))
return
}
endpoint := strings.TrimRight(site.BaseURL, "/") + "/wp-json/dsgt/v1/balance"
for _, ref := range refs {
payload, err := fetchBalance(ctx, sess, endpoint, ref.id)
w.record(site.Code, ref, route, payload, err, warnRatio)
}
}
func (w *Watcher) markAll(site string, refs []botRef, route string, err error) {
for _, ref := range refs {
w.record(site, ref, route, nil, err, 0)
}
}
func (w *Watcher) record(site string, ref botRef, route string, payload *balancePayload, err error, warnRatio float64) {
key := site + "|" + strconv.Itoa(ref.id)
b := &Balance{
Site: site, BotID: ref.id, Model: ref.model,
Route: routeLabel(route), CheckedAt: time.Now().Unix(),
}
if err != nil {
b.Error = err.Error()
} else {
b.PaidBalance = payload.Balance
b.Limit = payload.Free.Limit
b.Used = payload.Free.Used
b.Remaining = payload.Free.Remaining
b.ResetPeriod = payload.Free.ResetPeriod
if b.Limit > 0 {
b.Ratio = float64(b.Remaining) / float64(b.Limit)
}
}
var lowNow, recovered, resetSeen bool
var prevUsed int64
w.mu.Lock()
if prev := w.entries[key]; prev != nil && prev.Error == "" && err == nil && b.Used < prev.Used {
resetSeen = true
prevUsed = prev.Used
}
w.entries[key] = b
prevErr := w.lastErr[key]
w.lastErr[key] = b.Error
if err == nil && b.Limit > 0 {
low := float64(b.Remaining) <= warnRatio*float64(b.Limit)
if low && !w.warned[key] {
w.warned[key] = true
lowNow = true
}
if !low && w.warned[key] {
w.warned[key] = false
recovered = true
}
}
w.mu.Unlock()
if err != nil {
if b.Error != prevErr {
w.log.Warn("quota check failed",
"site", site, "bot_id", ref.id, "model", ref.model, "route", b.Route, "error", b.Error)
}
return
}
if prevErr != "" {
w.log.Info("quota check recovered", "site", site, "bot_id", ref.id, "route", b.Route)
}
w.log.Debug("quota check",
"site", site, "bot_id", ref.id, "model", ref.model, "route", b.Route,
"remaining", b.Remaining, "limit", b.Limit, "used", b.Used)
if resetSeen {
w.log.Info("quota reset observed",
"site", site, "bot_id", ref.id, "model", ref.model, "route", b.Route,
"used_before", prevUsed, "used_after", b.Used,
"checked_at", time.Unix(b.CheckedAt, 0).Format(time.RFC3339))
}
switch {
case lowNow:
w.log.Warn("quota low",
"site", site, "bot_id", ref.id, "model", ref.model, "route", b.Route,
"remaining", b.Remaining, "limit", b.Limit, "reset_period", b.ResetPeriod)
case recovered:
w.log.Info("quota recovered",
"site", site, "bot_id", ref.id, "model", ref.model,
"remaining", b.Remaining, "limit", b.Limit)
}
}
func (w *Watcher) routeFor(site string) string {
if w.pool != nil {
if cands := w.pool.Candidates(site); len(cands) > 0 {
return cands[0].Proxy
}
}
return ""
}
func (w *Watcher) session(proxy string) (httpx.Session, error) {
if w.newSession != nil {
return w.newSession(proxy)
}
return httpx.NewSession(httpx.Options{ProxyURL: proxy, Timeout: 45 * time.Second})
}
func routeLabel(proxy string) string {
if proxy == "" {
return "direct"
}
return proxy
}
// freeQuota is the daily guest allowance block of a balance response.
type freeQuota struct {
Limit int64 `json:"limit"`
Used int64 `json:"used"`
Remaining int64 `json:"remaining"`
ResetPeriod string `json:"reset_period"`
}
type balancePayload struct {
Balance int64 `json:"balance"`
Free *freeQuota `json:"free"`
}
func fetchBalance(ctx context.Context, sess httpx.Session, endpoint string, botID int) (*balancePayload, error) {
u := fmt.Sprintf("%s?bot_id=%d", endpoint, botID)
resp, err := sess.Do(ctx, httpx.Request{
Method: http.MethodGet,
URL: u,
Header: map[string]string{"Accept": "application/json"},
})
if err != nil {
return nil, fmt.Errorf("balance request: %w", err)
}
if resp.Status != http.StatusOK {
return nil, fmt.Errorf("balance request: HTTP %d", resp.Status)
}
var p balancePayload
if err := json.Unmarshal(resp.Body, &p); err != nil {
return nil, fmt.Errorf("balance json: %w", err)
}
if p.Free == nil {
return nil, errors.New("balance response has no free quota block")
}
return &p, nil
}
// Snapshot returns the current readings for the console.
func (w *Watcher) Snapshot() Snapshot {
w.cfg.RLock()
enabled := w.cfg.Quota.Enabled
checkS := w.cfg.Quota.CheckSecondsValue()
ratio := w.cfg.Quota.WarnRatioValue()
w.cfg.RUnlock()
w.mu.Lock()
entries := make([]Balance, 0, len(w.entries))
for _, b := range w.entries {
entries = append(entries, *b)
}
running := w.running
w.mu.Unlock()
sort.Slice(entries, func(i, j int) bool {
if entries[i].Site != entries[j].Site {
return entries[i].Site < entries[j].Site
}
return entries[i].BotID < entries[j].BotID
})
return Snapshot{Enabled: enabled, Running: running, CheckS: checkS, WarnRatio: ratio, Entries: entries}
}
+105
View File
@@ -0,0 +1,105 @@
package quota
import (
"bytes"
"context"
"log/slog"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"github.com/nyoungo/dsfree2api/internal/config"
"github.com/nyoungo/dsfree2api/internal/httpx"
"github.com/nyoungo/dsfree2api/internal/turnstile"
)
const examplePath = "../../config.example.toml"
func TestQuotaResetIsRecorded(t *testing.T) {
cfg, err := config.Load(examplePath)
if err != nil {
t.Fatalf("load config: %v", err)
}
var buf bytes.Buffer
w := New(cfg, slog.New(slog.NewTextHandler(&buf, nil)), nil, nil)
ref := botRef{id: 7, model: "m"}
w.record("de", ref, "", &balancePayload{Free: &freeQuota{Limit: 30000, Used: 500, Remaining: 29500, ResetPeriod: "daily"}}, nil, 0.2)
w.record("de", ref, "", &balancePayload{Free: &freeQuota{Limit: 30000, Used: 0, Remaining: 30000, ResetPeriod: "daily"}}, nil, 0.2)
if !strings.Contains(buf.String(), "quota reset observed") {
t.Fatalf("expected a reset log line, got:\n%s", buf.String())
}
if !strings.Contains(buf.String(), "used_before=500") {
t.Fatalf("expected used_before=500 in log, got:\n%s", buf.String())
}
}
func TestSweepReadsBalancesWithPooledCookies(t *testing.T) {
var seenGid string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if !strings.HasPrefix(r.URL.Path, "/wp-json/dsgt/v1/balance") {
t.Errorf("unexpected request %s — the sentinel must not fetch the site page", r.URL.Path)
return
}
if c, err := r.Cookie("dsgt_gid"); err == nil {
seenGid = c.Value
}
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"balance":0,"free":{"limit":30000,"used":500,"remaining":29500,"reset_period":"daily"}}`))
}))
defer srv.Close()
cfg, err := config.Load(examplePath)
if err != nil {
t.Fatalf("load config: %v", err)
}
cfg.Quota.Enabled = true
cfg.Quota.CheckSeconds = 300
cfg.Quota.WarnRatio = 0.2
site := *cfg.Sites["de"]
site.BaseURL = srv.URL
cfg.Sites["de"] = &site
ts := turnstile.New(cfg.Turnstile, "")
if _, _, err := ts.ImportCookies("de", "dsgt_gid=test-gid; cf_clearance=abc", ""); err != nil {
t.Fatalf("import cookies: %v", err)
}
w := New(cfg, nil, ts, nil)
w.newSession = func(string) (httpx.Session, error) {
return httpx.NewSession(httpx.Options{Timeout: 5 * time.Second})
}
w.Sweep(context.Background())
if seenGid != "test-gid" {
t.Fatalf("balance call visitor cookie = %q, want test-gid", seenGid)
}
snap := w.Snapshot()
if !snap.Enabled {
t.Error("snapshot should report enabled")
}
if len(snap.Entries) != 6 {
t.Fatalf("entries = %d, want 6 (3 sites × 2 bots)", len(snap.Entries))
}
var de *Balance
for i := range snap.Entries {
if snap.Entries[i].Site == "de" && snap.Entries[i].BotID == 27487 {
de = &snap.Entries[i]
}
}
if de == nil {
t.Fatalf("no de/27487 entry in %+v", snap.Entries)
}
if de.Error != "" || de.Limit != 30000 || de.Remaining != 29500 || de.Used != 500 || de.ResetPeriod != "daily" {
t.Fatalf("de balance = %+v", de)
}
if de.Route != "direct" {
t.Errorf("route = %q, want direct", de.Route)
}
for _, e := range snap.Entries {
if e.Site != "de" && e.Error == "" {
t.Errorf("site %s should report missing cookies, got %+v", e.Site, e)
}
}
}
+50 -9
View File
@@ -467,10 +467,7 @@ const uaMetadataExpr = `(() => {
// warms the page like a human, renders the widget and clicks until a token
// lands or the deadline passes.
func (s *Solver) pollForToken(ctx context.Context, conn *cdp, sid, sitekey string, cfg config.Turnstile) (string, error) {
timeout := time.Duration(cfg.TimeoutSeconds) * time.Second
if timeout <= 0 {
timeout = 90 * time.Second
}
timeout := time.Duration(cfg.SolveTimeoutValue()) * time.Second
deadline := time.Now().Add(timeout)
var preDone, inited bool
@@ -941,6 +938,10 @@ func launchBrowser(ctx context.Context, exe string, cfg config.Turnstile, locale
"--user-data-dir=" + profile,
"--no-first-run",
"--no-default-browser-check",
// Chromium/Edge re-launch themselves de-elevated when the parent runs
// as Administrator, orphaning the real browser and breaking CDP;
// keep the browser inside our process tree instead.
"--do-not-de-elevate",
"--disable-blink-features=AutomationControlled",
"--disable-background-networking",
"--disable-sync",
@@ -992,13 +993,12 @@ func (b *browserProc) waitDevTools(ctx context.Context) error {
deadline := time.Now().Add(30 * time.Second)
var lastErr error
for time.Now().Before(deadline) {
select {
case <-b.exited:
return fmt.Errorf("browser exited early (%v): %s", b.waitErr, b.stderr.String())
case <-ctx.Done():
if ctx.Err() != nil {
return ctx.Err()
default:
}
// A Windows browser launcher may hand off to a longer-lived process
// and exit 0; that is not a failure while the DevTools endpoint is
// still coming up, so the exit check happens only at timeout.
resp, err := client.Get(endpoint)
if err == nil {
body, _ := io.ReadAll(resp.Body)
@@ -1018,10 +1018,17 @@ func (b *browserProc) waitDevTools(ctx context.Context) error {
return ctx.Err()
}
}
select {
case <-b.exited:
return fmt.Errorf("browser exited early (%v): %s", b.waitErr, b.stderr.String())
default:
}
return fmt.Errorf("devtools never came up: %v; stderr: %s", lastErr, b.stderr.String())
}
// Close kills the browser if it is still alive and drops the temp profile.
// On Windows the spawned launcher may have exited while the real browser
// lives on, so a DevTools close is attempted before giving up.
func (b *browserProc) Close() {
select {
case <-b.exited:
@@ -1034,6 +1041,7 @@ func (b *browserProc) Close() {
case <-time.After(3 * time.Second):
}
}
b.closeViaDevTools(2 * time.Second)
if b.tempDir != "" {
for i := 0; i < 3; i++ {
if err := os.RemoveAll(b.tempDir); err == nil {
@@ -1044,6 +1052,39 @@ func (b *browserProc) Close() {
}
}
// closeViaDevTools asks the browser to shut down over the DevTools endpoint.
// It is the only handle left when the original launcher process has exited.
func (b *browserProc) closeViaDevTools(timeout time.Duration) {
ws := b.wsURL
if ws == "" {
client := &http.Client{Timeout: timeout}
resp, err := client.Get(fmt.Sprintf("http://127.0.0.1:%d/json/version", b.port))
if err != nil {
return
}
body, _ := io.ReadAll(resp.Body)
resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return
}
var info struct {
WebSocketDebuggerURL string `json:"webSocketDebuggerUrl"`
}
if json.Unmarshal(body, &info) != nil || info.WebSocketDebuggerURL == "" {
return
}
ws = info.WebSocketDebuggerURL
}
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
conn, err := dialCDP(ctx, ws, fmt.Sprintf("http://127.0.0.1:%d", b.port))
if err != nil {
return
}
defer conn.Close()
_, _ = conn.call(ctx, "", "Browser.close", nil)
}
func freePort() (int, error) {
l, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
+85 -13
View File
@@ -10,7 +10,9 @@ import (
"errors"
"fmt"
"io"
"net"
"net/http"
"net/url"
"strings"
"sync"
"time"
@@ -91,6 +93,11 @@ type Solver struct {
// browserMu serializes browser solves: one Chrome window at a time.
browserMu sync.Mutex
// cookie pool: background warm refresh, one slot per site × route.
poolMu sync.Mutex
poolOn bool
warms map[string]*warmState
// refreshOverride is a test seam; nil in production.
refreshOverride func(ctx context.Context, sess httpx.Session, site *config.Site, c *core) error
}
@@ -107,7 +114,7 @@ func New(cfg config.Turnstile, userAgent string) *Solver {
}
}
func (s *Solver) Enabled() bool { return s.cfg.Enabled }
func (s *Solver) Enabled() bool { return s.Config().Enabled }
func (s *Solver) Config() config.Turnstile {
s.mu.Lock()
@@ -133,7 +140,7 @@ func (s *Solver) coreFor(proxy string) *core {
defer s.mu.Unlock()
c, ok := s.cores[key]
if !ok {
client, _ := httpx.StdClient(proxy, time.Duration(s.cfg.TimeoutSeconds+30)*time.Second)
client, _ := httpx.StdClient(proxy, time.Duration(s.cfg.SolveTimeoutValue()+30)*time.Second)
c = &core{
proxy: key,
client: client,
@@ -287,9 +294,17 @@ func (s *Solver) ApplyValidCookies(ctx context.Context, sess httpx.Session, site
return s.refreshLocked(ctx, sess, site, c)
}
// CookiesReady reports whether an unexpired cached cookie set exists for the
// site × route pair. Upstream uses it to keep the slow-start timer from
// killing a first-time Turnstile solve, which can legitimately take minutes.
func (s *Solver) CookiesReady(siteCode, proxy string) bool {
c := s.coreFor(proxy)
return c.load(siteCode) != nil
}
// Refresh forces a fresh solve → verify cycle.
func (s *Solver) Refresh(ctx context.Context, sess httpx.Session, site *config.Site, proxy string) error {
if !s.cfg.Enabled {
if !s.Enabled() {
return nil
}
c := s.coreFor(proxy)
@@ -385,32 +400,70 @@ func (s *Solver) solveToken(ctx context.Context, site *config.Site, sitekey stri
func (s *Solver) solveTokenAPI(ctx context.Context, baseURL, sitekey string, c *core) (string, error) {
cfg := s.Config()
payload, _ := json.Marshal(map[string]any{
"url": baseURL,
"sitekey": sitekey,
"action": cfg.Action,
"cdata": "",
"timeoutSeconds": cfg.TimeoutSeconds,
})
style := cfg.APIStyleValue()
var payload []byte
if style == "ezsolver" {
payload, _ = json.Marshal(map[string]any{
"sitekey": sitekey,
"siteurl": baseURL,
"timeout": cfg.SolveTimeoutValue(),
})
} else {
payload, _ = json.Marshal(map[string]any{
"url": baseURL,
"sitekey": sitekey,
"action": cfg.Action,
"cdata": "",
"timeoutSeconds": cfg.SolveTimeoutValue(),
})
}
var last error
// A loopback solve service must be reached directly: pushing it through
// the route proxy would resolve 127.0.0.1 on the far side of the tunnel.
client := c.client
if isLocalHost(cfg.APIURL) {
client = &http.Client{Timeout: time.Duration(cfg.SolveTimeoutValue()+30) * time.Second}
}
for attempt := 0; attempt < cfg.Retries; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, cfg.APIURL, bytes.NewReader(payload))
if err != nil {
return "", err
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Authorization", "Bearer "+cfg.APIKey)
resp, err := c.client.Do(req)
if cfg.APIKey != "" {
req.Header.Set("Authorization", "Bearer "+cfg.APIKey)
}
resp, err := client.Do(req)
if err != nil {
last = &Error{Msg: "turnstile solver request failed: " + err.Error(), Retryable: true}
} else {
body, _ := io.ReadAll(resp.Body)
resp.Body.Close()
if resp.StatusCode >= 400 {
retry := retryableStatus(resp.StatusCode)
if style == "ezsolver" {
// Local solve services answer 5xx for per-attempt
// timeouts; another attempt is reasonable.
retry = true
}
last = &Error{
Msg: fmt.Sprintf("turnstile solver http %d: %s (check [turnstile].api_key / api_url)",
resp.StatusCode, trim(body)),
Retryable: retryableStatus(resp.StatusCode),
Retryable: retry,
}
} else if style == "ezsolver" {
var out struct {
Token string `json:"token"`
Error string `json:"error"`
}
if err := json.Unmarshal(body, &out); err != nil {
last = &Error{Msg: "turnstile solver bad json: " + err.Error()}
} else if out.Error != "" {
last = &Error{Msg: "turnstile solver error: " + trim([]byte(out.Error))}
} else if out.Token == "" {
last = &Error{Msg: "turnstile solver returned no token"}
} else {
return out.Token, nil
}
} else {
var out struct {
@@ -444,6 +497,25 @@ func (s *Solver) solveTokenAPI(ctx context.Context, baseURL, sitekey string, c *
return "", last
}
// isLocalHost reports whether rawURL points at the local machine or a
// private-network address. Such solve services must be reached directly
// instead of through the route proxy.
func isLocalHost(rawURL string) bool {
u, err := url.Parse(rawURL)
if err != nil {
return false
}
host := strings.ToLower(u.Hostname())
if host == "localhost" {
return true
}
ip := net.ParseIP(host)
if ip == nil {
return false
}
return ip.IsLoopback() || ip.IsPrivate() || ip.IsLinkLocalUnicast() || ip.IsUnspecified()
}
// retryableStatus decides which solver HTTP statuses are worth another attempt.
// Solver-side rejections (bad key, exhausted quota, challenge rejected) return
// the same answer every time, so they are not retried.
+44
View File
@@ -2,7 +2,11 @@ package turnstile
import (
"context"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"net/url"
"reflect"
"strings"
"sync"
@@ -239,3 +243,43 @@ func TestRotateGuestSwapsVisitorID(t *testing.T) {
t.Errorf("rotating an unknown site returned %q, want empty", got)
}
}
func TestSolveTokenAPIEzSolverStyle(t *testing.T) {
var gotBody map[string]any
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if err := json.NewDecoder(r.Body).Decode(&gotBody); err != nil {
t.Errorf("decode request: %v", err)
}
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"token":"tok-123","elapsed":1.5}`))
}))
defer srv.Close()
cfg := config.Turnstile{
Enabled: true, Provider: config.ProviderAPI, APIStyle: "ezsolver",
APIURL: srv.URL, TimeoutSeconds: 30, Retries: 1,
}
solver := New(cfg, httpx.DefaultUserAgent)
// A broken route-proxied client proves the loopback bypass: the call must
// still succeed because local solve services are reached directly.
brokenURL, _ := url.Parse("socks5://127.0.0.1:1")
c := &core{
proxy: "direct",
client: &http.Client{Transport: &http.Transport{Proxy: http.ProxyURL(brokenURL)}},
cache: map[string]*cookieState{}, locks: map[string]*sync.Mutex{},
}
token, err := solver.solveTokenAPI(context.Background(), "https://deepseek.de/", "0xabc", c)
if err != nil {
t.Fatalf("solve: %v", err)
}
if token != "tok-123" {
t.Fatalf("token = %q", token)
}
if gotBody["sitekey"] != "0xabc" || gotBody["siteurl"] != "https://deepseek.de/" {
t.Fatalf("request body = %v", gotBody)
}
if gotBody["timeout"] != float64(30) {
t.Fatalf("timeout = %v", gotBody["timeout"])
}
}
+278
View File
@@ -0,0 +1,278 @@
package turnstile
import (
"context"
"sort"
"time"
"github.com/nyoungo/dsfree2api/internal/config"
"github.com/nyoungo/dsfree2api/internal/httpx"
)
// WarmTarget is one cookie-pool slot: a site reached through one proxy route.
// Verified cookies are bound to the egress IP that solved them, so the pool
// keeps one slot per site × route.
type WarmTarget struct {
Site config.Site
Proxy string // "" means direct
}
// WarmEntry describes the pool state of one site × route slot.
type WarmEntry struct {
Site string `json:"site"`
Route string `json:"route"`
Valid bool `json:"valid"`
Remaining int64 `json:"remaining_s"`
Warming bool `json:"warming"`
LastAt int64 `json:"last_at"`
LastMs int64 `json:"last_ms"`
LastErr string `json:"last_error,omitempty"`
NextWarm int64 `json:"next_warm_at"`
}
// PoolStatus is the console view of the cookie pool.
type PoolStatus struct {
Enabled bool `json:"enabled"`
Running bool `json:"running"`
Ratio float64 `json:"ratio"`
CheckS int `json:"check_seconds"`
Entries []WarmEntry `json:"entries"`
}
type warmState struct {
site string
route string
warming bool
fails int
lastAt time.Time
lastMs int64
lastErr string
}
// StartWarmer launches the cookie-pool loop. targets is consulted on every
// sweep, so console edits (sites, routes, toggles) are picked up live. A slot
// is warmed when its cookie is missing, expired, or has less than warm_ratio
// of its TTL left — requests then never wait for a solve to finish.
func (s *Solver) StartWarmer(ctx context.Context, targets func() []WarmTarget) {
if targets == nil {
return
}
s.poolMu.Lock()
if s.poolOn {
s.poolMu.Unlock()
return
}
s.poolOn = true
if s.warms == nil {
s.warms = map[string]*warmState{}
}
s.poolMu.Unlock()
go s.warmLoop(ctx, targets)
}
func (s *Solver) warmLoop(ctx context.Context, targets func() []WarmTarget) {
defer func() {
s.poolMu.Lock()
s.poolOn = false
s.poolMu.Unlock()
}()
for {
s.sweep(ctx, targets)
wait := time.Duration(s.Config().WarmCheckValue()) * time.Second
select {
case <-ctx.Done():
return
case <-time.After(wait):
}
}
}
// sweep warms every due slot. Solves run in the background so one slow
// browser window cannot stall the inspection; a slot already being warmed is
// skipped until it finishes, and a slot whose last attempt failed waits out an
// exponential backoff before the next try.
func (s *Solver) sweep(ctx context.Context, targets func() []WarmTarget) {
cfg := s.Config()
if !cfg.Enabled || !cfg.WarmEnabled || cfg.ProviderValue() == config.ProviderManual {
return
}
ttl := time.Duration(cfg.CookieTTLSeconds) * time.Second
if ttl <= 0 {
ttl = 3 * time.Hour
}
for _, tgt := range targets() {
key := poolKey(tgt.Proxy, tgt.Site.Code)
if !s.warmDue(key, tgt, ttl, cfg.WarmRatioValue(), cfg.WarmCheckValue()) {
continue
}
s.markWarm(key, tgt, true, time.Time{}, 0, "")
go s.warmOne(ctx, key, tgt, cfg)
}
}
func (s *Solver) warmDue(key string, tgt WarmTarget, ttl time.Duration, ratio float64, checkS int) bool {
s.poolMu.Lock()
st := s.warms[key]
if st != nil && st.warming {
s.poolMu.Unlock()
return false
}
var lastAt time.Time
fails := 0
if st != nil {
lastAt, fails = st.lastAt, st.fails
}
s.poolMu.Unlock()
// A failed slot backs off exponentially (×2 per consecutive failure, up
// to 15 minutes) so a hopeless route cannot retry-storm the browser pool.
if fails > 0 && !lastAt.IsZero() {
backoff := time.Duration(checkS) * time.Second
for i := 1; i < fails && backoff < 15*time.Minute; i++ {
backoff *= 2
}
if backoff > 15*time.Minute || backoff <= 0 {
backoff = 15 * time.Minute
}
if time.Since(lastAt) < backoff {
return false
}
}
c := s.coreFor(tgt.Proxy)
c.mu.Lock()
cs := c.cache[tgt.Site.Code]
c.mu.Unlock()
if cs == nil {
return true // cold slot — fill it
}
return time.Until(cs.expiresAt) <= time.Duration(ratio*float64(ttl))
}
func (s *Solver) warmOne(ctx context.Context, key string, tgt WarmTarget, cfg config.Turnstile) {
site := tgt.Site
start := time.Now()
sess, err := httpx.NewSession(httpx.Options{
ProxyURL: tgt.Proxy,
Timeout: time.Duration(cfg.SolveTimeoutValue()+30) * time.Second,
})
if err == nil {
err = s.Refresh(ctx, sess, &site, tgt.Proxy)
sess.Close()
}
lastErr := ""
if err != nil {
lastErr = err.Error()
}
s.markWarm(key, tgt, false, start, time.Since(start).Milliseconds(), lastErr)
}
func (s *Solver) markWarm(key string, tgt WarmTarget, warming bool, at time.Time, ms int64, lastErr string) {
s.poolMu.Lock()
defer s.poolMu.Unlock()
if s.warms == nil {
s.warms = map[string]*warmState{}
}
st, ok := s.warms[key]
if !ok {
st = &warmState{site: tgt.Site.Code, route: routeLabel(tgt.Proxy)}
s.warms[key] = st
}
st.warming = warming
if !at.IsZero() {
st.lastAt = at
st.lastMs = ms
st.lastErr = lastErr
if lastErr == "" {
st.fails = 0
} else {
st.fails++
}
}
}
// PoolStatus reports every known slot: cached lifetime, warm progress, last
// solve duration and the next planned refresh.
func (s *Solver) PoolStatus() PoolStatus {
cfg := s.Config()
pooling := cfg.Enabled && cfg.WarmEnabled && cfg.ProviderValue() != config.ProviderManual
ps := PoolStatus{
Enabled: pooling,
Ratio: cfg.WarmRatioValue(),
CheckS: cfg.WarmCheckValue(),
Entries: make([]WarmEntry, 0, 4),
}
s.poolMu.Lock()
ps.Running = s.poolOn
states := make(map[string]*warmState, len(s.warms))
for k, v := range s.warms {
cp := *v
states[k] = &cp
}
s.poolMu.Unlock()
s.mu.Lock()
cores := make([]*core, 0, len(s.cores))
for _, c := range s.cores {
cores = append(cores, c)
}
s.mu.Unlock()
now := time.Now()
ttl := time.Duration(cfg.CookieTTLSeconds) * time.Second
if ttl <= 0 {
ttl = 3 * time.Hour
}
entries := map[string]*WarmEntry{}
for _, c := range cores {
c.mu.Lock()
for code, cs := range c.cache {
e := &WarmEntry{Site: code, Route: c.proxy, Valid: now.Before(cs.expiresAt)}
if e.Valid {
e.Remaining = int64(cs.expiresAt.Sub(now).Seconds())
}
entries[poolKey(c.proxy, code)] = e
}
c.mu.Unlock()
}
for k, st := range states {
e, ok := entries[k]
if !ok {
e = &WarmEntry{Site: st.site, Route: st.route}
entries[k] = e
}
e.Warming = st.warming
if !st.lastAt.IsZero() {
e.LastAt = st.lastAt.Unix()
}
e.LastMs = st.lastMs
e.LastErr = st.lastErr
}
for _, e := range entries {
if pooling && e.Valid {
until := time.Duration(e.Remaining)*time.Second - time.Duration(ps.Ratio*float64(ttl))
if until < 0 {
until = 0
}
e.NextWarm = now.Add(until).Unix()
}
ps.Entries = append(ps.Entries, *e)
}
sort.Slice(ps.Entries, func(i, j int) bool {
if ps.Entries[i].Site != ps.Entries[j].Site {
return ps.Entries[i].Site < ps.Entries[j].Site
}
return ps.Entries[i].Route < ps.Entries[j].Route
})
return ps
}
func poolKey(proxy, site string) string { return routeLabel(proxy) + "|" + site }
func routeLabel(proxy string) string {
if proxy == "" {
return "direct"
}
return proxy
}
+206
View File
@@ -0,0 +1,206 @@
package turnstile
import (
"context"
"sync/atomic"
"testing"
"time"
"github.com/nyoungo/dsfree2api/internal/config"
"github.com/nyoungo/dsfree2api/internal/httpx"
)
func warmerSolver(t *testing.T, mutate func(*config.Turnstile)) (*Solver, *config.Config) {
t.Helper()
cfg, err := config.Load(examplePath)
if err != nil {
t.Fatalf("load config: %v", err)
}
cfg.Turnstile.Enabled = true
cfg.Turnstile.WarmEnabled = true
cfg.Turnstile.CookieTTLSeconds = 3600
cfg.Turnstile.WarmRatio = 0.3
if mutate != nil {
mutate(&cfg.Turnstile)
}
return New(cfg.Turnstile, httpx.DefaultUserAgent), cfg
}
func waitCalls(t *testing.T, calls *int32, want int32) {
t.Helper()
deadline := time.Now().Add(2 * time.Second)
for time.Now().Before(deadline) {
if atomic.LoadInt32(calls) >= want {
return
}
time.Sleep(10 * time.Millisecond)
}
t.Fatalf("refresh calls = %d, want >= %d", atomic.LoadInt32(calls), want)
}
func waitSettled(t *testing.T, solver *Solver) {
t.Helper()
deadline := time.Now().Add(2 * time.Second)
for time.Now().Before(deadline) {
ps := solver.PoolStatus()
busy := false
for _, e := range ps.Entries {
if e.Warming {
busy = true
}
}
if !busy {
return
}
time.Sleep(10 * time.Millisecond)
}
t.Fatal("cookie pool did not settle")
}
// rewindLastAt backdates the last attempt of a slot so backoff windows can be
// tested without sleeping through them.
func rewindLastAt(solver *Solver, key string, d time.Duration) {
solver.poolMu.Lock()
if st := solver.warms[key]; st != nil {
st.lastAt = time.Now().Add(-d)
}
solver.poolMu.Unlock()
}
func TestWarmDueBacksOffAfterFailure(t *testing.T) {
solver, cfg := warmerSolver(t, nil)
tgt := WarmTarget{Site: *cfg.Sites["de"], Proxy: ""}
key := poolKey("", "de")
ttl := time.Hour
if !solver.warmDue(key, tgt, ttl, 0.3, 5) {
t.Fatal("cold slot should be due")
}
// First failure: the 5s backoff must hold.
solver.markWarm(key, tgt, false, time.Now(), 100, "boom")
if solver.warmDue(key, tgt, ttl, 0.3, 5) {
t.Fatal("first backoff was ignored")
}
// Second failure doubles it: 6s elapsed is still inside the 10s window.
solver.markWarm(key, tgt, false, time.Now(), 100, "boom")
rewindLastAt(solver, key, 6*time.Second)
if solver.warmDue(key, tgt, ttl, 0.3, 5) {
t.Fatal("second failure did not double the backoff")
}
// 11s elapsed clears the 10s window.
rewindLastAt(solver, key, 11*time.Second)
if !solver.warmDue(key, tgt, ttl, 0.3, 5) {
t.Fatal("slot should be due once the backoff elapsed")
}
// A success resets the failure counter.
solver.markWarm(key, tgt, false, time.Now(), 50, "")
if !solver.warmDue(key, tgt, ttl, 0.3, 5) {
t.Fatal("success should clear the backoff")
}
// The backoff is capped at 15 minutes.
for i := 0; i < 12; i++ {
solver.markWarm(key, tgt, false, time.Now(), 100, "boom")
}
rewindLastAt(solver, key, 11*time.Second)
if solver.warmDue(key, tgt, ttl, 0.3, 5) {
t.Fatal("capped backoff should still hold at 11s")
}
rewindLastAt(solver, key, 16*time.Minute)
if !solver.warmDue(key, tgt, ttl, 0.3, 5) {
t.Fatal("slot should be due after the capped backoff")
}
}
func TestWarmSweepFillsColdSlotAndSkipsFresh(t *testing.T) {
solver, cfg := warmerSolver(t, nil)
var calls int32
solver.refreshOverride = func(_ context.Context, _ httpx.Session, s *config.Site, c *core) error {
atomic.AddInt32(&calls, 1)
c.mu.Lock()
c.cache[s.Code] = &cookieState{cookies: map[string]string{"dsts": "ok"}, expiresAt: time.Now().Add(time.Hour)}
c.mu.Unlock()
return nil
}
targets := func() []WarmTarget {
return []WarmTarget{{Site: *cfg.Sites["de"], Proxy: ""}}
}
// Cold slot: the first sweep must start a solve.
solver.sweep(context.Background(), targets)
waitCalls(t, &calls, 1)
waitSettled(t, solver)
// Fresh cookie (1h left, threshold 0.3×1h): a second sweep must not solve.
solver.sweep(context.Background(), targets)
time.Sleep(80 * time.Millisecond)
if got := atomic.LoadInt32(&calls); got != 1 {
t.Fatalf("fresh slot was re-warmed: calls = %d", got)
}
// Remaining TTL below the threshold: due again.
c := solver.coreFor("")
c.mu.Lock()
c.cache["de"] = &cookieState{cookies: map[string]string{"dsts": "ok"}, expiresAt: time.Now().Add(30 * time.Second)}
c.mu.Unlock()
solver.sweep(context.Background(), targets)
waitCalls(t, &calls, 2)
waitSettled(t, solver)
ps := solver.PoolStatus()
if !ps.Enabled {
t.Error("pool should report enabled")
}
if len(ps.Entries) != 1 {
t.Fatalf("pool entries = %d, want 1", len(ps.Entries))
}
e := ps.Entries[0]
if !e.Valid || e.LastAt == 0 || e.LastErr != "" || e.Warming {
t.Errorf("pool entry = %+v", e)
}
if e.NextWarm == 0 {
t.Error("next_warm_at should be scheduled for a valid pooled cookie")
}
}
func TestWarmSweepSkipsManualAndDisabled(t *testing.T) {
var calls int32
override := func(context.Context, httpx.Session, *config.Site, *core) error {
atomic.AddInt32(&calls, 1)
return nil
}
solver, cfg := warmerSolver(t, func(tc *config.Turnstile) { tc.Provider = config.ProviderManual })
solver.refreshOverride = override
solver.sweep(context.Background(), func() []WarmTarget {
return []WarmTarget{{Site: *cfg.Sites["de"]}}
})
time.Sleep(50 * time.Millisecond)
if calls != 0 {
t.Fatalf("manual provider must not warm, calls = %d", calls)
}
solver2, cfg2 := warmerSolver(t, func(tc *config.Turnstile) { tc.Enabled = false })
solver2.refreshOverride = override
solver2.sweep(context.Background(), func() []WarmTarget {
return []WarmTarget{{Site: *cfg2.Sites["de"]}}
})
time.Sleep(50 * time.Millisecond)
if calls != 0 {
t.Fatalf("disabled solver must not warm, calls = %d", calls)
}
solver3, cfg3 := warmerSolver(t, func(tc *config.Turnstile) { tc.WarmEnabled = false })
solver3.refreshOverride = override
solver3.sweep(context.Background(), func() []WarmTarget {
return []WarmTarget{{Site: *cfg3.Sites["de"]}}
})
time.Sleep(50 * time.Millisecond)
if calls != 0 {
t.Fatalf("pool switch off must not warm, calls = %d", calls)
}
}
+82 -6
View File
@@ -35,6 +35,19 @@ func (r Route) Key() string {
return r.Proxy
}
// RouteCandidate is one egress route supplied by the proxy pool.
type RouteCandidate struct {
Name string
Proxy string
}
// RouteSource resolves per-site route candidates and records request
// outcomes. Implemented by proxypool.Manager; nil when the pool is unused.
type RouteSource interface {
Candidates(site string) []RouteCandidate
Report(proxy string, ok bool)
}
type chatConfig struct {
BotID int
PostID int
@@ -44,9 +57,10 @@ type chatConfig struct {
}
type Client struct {
cfg *config.Config
ts *turnstile.Solver
log *slog.Logger
cfg *config.Config
ts *turnstile.Solver
log *slog.Logger
pool RouteSource
stateMu sync.Mutex
chatCfg map[string]chatConfig
@@ -61,12 +75,17 @@ type Client struct {
lastWarnMsg string
lastWarnAt time.Time
// quotaObserver, when set, is notified on the first site-side quota
// exhaustion of each request — including ones hidden by cross-site
// failover.
quotaObserver func(site, model, route string, err error)
// chatOnceOverride is a test seam; nil in production.
chatOnceOverride func(ctx context.Context, site config.Site, modelID string, model config.Model,
prompt string, route Route, info *ServeInfo, yield func(Event) error) error
}
func New(cfg *config.Config, ts *turnstile.Solver, log *slog.Logger) *Client {
func New(cfg *config.Config, ts *turnstile.Solver, log *slog.Logger, pool RouteSource) *Client {
if log == nil {
log = slog.Default()
}
@@ -74,6 +93,7 @@ func New(cfg *config.Config, ts *turnstile.Solver, log *slog.Logger) *Client {
cfg: cfg,
ts: ts,
log: log,
pool: pool,
chatCfg: map[string]chatConfig{},
cfgLocks: map[string]*sync.Mutex{},
gates: map[string]chan struct{}{},
@@ -81,6 +101,13 @@ func New(cfg *config.Config, ts *turnstile.Solver, log *slog.Logger) *Client {
}
}
// SetQuotaObserver registers a callback that fires whenever a site reports
// quota exhaustion, even when cross-site failover later succeeds and the error
// would otherwise never surface. Set it before serving requests.
func (c *Client) SetQuotaObserver(fn func(site, model, route string, err error)) {
c.quotaObserver = fn
}
// Routes returns the primary proxy followed by the fallbacks, deduplicated.
func (c *Client) Routes() []Route {
c.cfg.RLock()
@@ -241,6 +268,40 @@ func (c *Client) logCacheReject(site, modelID, prompt string) {
"head", strconv.Quote(head), "tail", strconv.Quote(tail))
}
// routesFor returns the ordered candidate routes for one site: proxy-pool
// candidates first (sticky, health filtered), then the global primary and
// fallback lines as the safety net.
func (c *Client) routesFor(siteCode string) []Route {
out := make([]Route, 0, 4)
seen := map[string]bool{}
add := func(r Route) {
key := r.Key()
if seen[key] {
return
}
seen[key] = true
out = append(out, r)
}
if c.pool != nil {
for _, cand := range c.pool.Candidates(siteCode) {
add(Route{Name: cand.Name, Proxy: cand.Proxy})
}
}
for _, r := range c.Routes() {
add(r)
}
return out
}
// poolReport feeds a route outcome back to the proxy pool; it is a no-op for
// global routes and the direct connection.
func (c *Client) poolReport(proxy string, ok bool) {
if c.pool == nil || proxy == "" {
return
}
c.pool.Report(proxy, ok)
}
// Chat streams the prompt through the upstream site, honouring route
// failover, slow-start detection, session refresh and cross-site failover.
func (c *Client) Chat(ctx context.Context, modelID string, prompt string, info *ServeInfo, yield func(Event) error) error {
@@ -357,23 +418,33 @@ func (c *Client) chatWithRetries(ctx context.Context, modelID string, model conf
if !site.Enabled {
return errf("site %s is disabled", model.Site)
}
routes := c.Routes()
routes := c.routesFor(site.Code)
quotaNotified := false
var lastErr error
sawQuota := false
for attempt := 0; ; attempt++ {
var failed []Route
for i, route := range routes {
allowSlow := i == 0 && len(routes) > 1
// A first-time Turnstile solve can far exceed the slow-start
// budget; only arm the timer when cookies are already cached
// (or no solver is configured at all).
cookiesReady := c.ts == nil || c.ts.CookiesReady(site.Code, route.Proxy)
allowSlow := i == 0 && len(routes) > 1 && cookiesReady
err := c.chatOnceSlowStart(ctx, site, modelID, model, prompt, route, allowSlow, info, yield)
if err == nil {
if sawQuota {
c.clearQuotaCool(site.Code)
c.probeBalanceAsync(site, model, route)
}
c.poolReport(route.Proxy, true)
return nil
}
lastErr = err
if isQuota(err) && !quotaNotified && c.quotaObserver != nil {
quotaNotified = true
c.quotaObserver(site.Code, modelID, route.Name, err)
}
if contentStarted(err) {
return AsUpstream(err)
}
@@ -400,6 +471,11 @@ func (c *Client) chatWithRetries(ctx context.Context, modelID string, model conf
if isCacheEmptyProblem(err) {
return AsUpstream(err)
}
// Quota/session failures are site-side, not proxy-side — do not
// blame the egress endpoint for them.
if !isQuota(err) && !isSession(err) {
c.poolReport(route.Proxy, false)
}
failed = append(failed, route)
c.logUpstreamErr(site.Code, modelID, attempt, route, err)
if isQuota(err) {
+35 -9
View File
@@ -32,7 +32,7 @@ func TestRoutesKeepPrimaryFirstAndDedupe(t *testing.T) {
cfg.Proxy.URL = ""
cfg.Proxy.FallbackURLs = []string{"socks5://fallback:1", "socks5://fallback:1"}
c := New(cfg, nil, nil)
c := New(cfg, nil, nil, nil)
got := c.Routes()
want := []Route{{Name: "primary", Proxy: ""}, {Name: "fallback-1", Proxy: "socks5://fallback:1"}}
if !reflect.DeepEqual(got, want) {
@@ -47,7 +47,7 @@ func TestSlowPrimarySwitchesToFallbackBeforeContent(t *testing.T) {
cfg.Proxy.SlowStartSeconds = 0.02
var calls []string
c := New(cfg, nil, nil)
c := New(cfg, nil, nil, nil)
c.chatOnceOverride = func(ctx context.Context, site config.Site, modelID string, model config.Model,
prompt string, route Route, info *ServeInfo, yield func(Event) error) error {
calls = append(calls, route.Name)
@@ -86,7 +86,7 @@ func TestPrimaryErrorSwitchesToFallbackBeforeContent(t *testing.T) {
cfg.Proxy.FallbackURLs = []string{"socks5://fallback:1"}
var calls []string
c := New(cfg, nil, nil)
c := New(cfg, nil, nil, nil)
c.chatOnceOverride = func(ctx context.Context, site config.Site, modelID string, model config.Model,
prompt string, route Route, info *ServeInfo, yield func(Event) error) error {
calls = append(calls, route.Name)
@@ -119,7 +119,7 @@ func TestNoFallbackAfterContentStarted(t *testing.T) {
cfg.Proxy.FallbackURLs = []string{"socks5://fallback:1"}
var calls []string
c := New(cfg, nil, nil)
c := New(cfg, nil, nil, nil)
c.chatOnceOverride = func(ctx context.Context, site config.Site, modelID string, model config.Model,
prompt string, route Route, info *ServeInfo, yield func(Event) error) error {
calls = append(calls, route.Name)
@@ -144,6 +144,32 @@ func TestNoFallbackAfterContentStarted(t *testing.T) {
}
}
func TestQuotaObserverFiresOncePerSiteBeforeFailover(t *testing.T) {
cfg := testConfig(t)
cfg.Upstream.AutoRefresh = false
c := New(cfg, nil, nil, nil)
c.chatOnceOverride = func(ctx context.Context, site config.Site, modelID string, model config.Model,
prompt string, route Route, info *ServeInfo, yield func(Event) error) error {
if site.Code == "de" {
return newQuota(`sse quota exhausted: {"quota_notice":{}}`)
}
yield(Event{Kind: KindDelta, Value: "ok"})
return nil
}
var fired []string
c.SetQuotaObserver(func(site, model, route string, err error) {
fired = append(fired, site+"|"+model+"|"+route)
})
if err := c.Chat(context.Background(), "deepseek-v4-flash-de", "hi", &ServeInfo{},
func(ev Event) error { return nil }); err != nil {
t.Fatalf("chat should succeed via cross-site failover: %v", err)
}
if len(fired) != 1 || fired[0] != "de|deepseek-v4-flash-de|primary" {
t.Fatalf("quota observer events = %v, want [de|deepseek-v4-flash-de|primary]", fired)
}
}
func TestQuotaNoticeTriggersQuotaError(t *testing.T) {
payload := map[string]any{
"error": "Dein heutiges Guthaben von 10.000 Token (V4-Pro) ist aufgebraucht.",
@@ -211,7 +237,7 @@ func TestTurnstileRequiredWhileDisabledFailsFast(t *testing.T) {
cfg.Upstream.RefreshRetries = 4
var calls int
c := New(cfg, nil, nil)
c := New(cfg, nil, nil, nil)
c.chatOnceOverride = func(ctx context.Context, site config.Site, modelID string, model config.Model,
prompt string, route Route, info *ServeInfo, yield func(Event) error) error {
calls++
@@ -237,7 +263,7 @@ func TestNonceFetchFailureFailsFast(t *testing.T) {
cfg.Upstream.RefreshRetries = 4
var calls int
c := New(cfg, nil, nil)
c := New(cfg, nil, nil, nil)
c.chatOnceOverride = func(ctx context.Context, site config.Site, modelID string, model config.Model,
prompt string, route Route, info *ServeInfo, yield func(Event) error) error {
calls++
@@ -273,7 +299,7 @@ func TestSSEDeltaAndDoneEvents(t *testing.T) {
func TestQuotaSiteCoolsDownAndPrefersSibling(t *testing.T) {
cfg := testConfig(t)
c := New(cfg, nil, nil)
c := New(cfg, nil, nil, nil)
calls := map[string]int{}
c.chatOnceOverride = func(_ context.Context, site config.Site, modelID string, model config.Model,
_ string, _ Route, _ *ServeInfo, _ func(Event) error) error {
@@ -318,7 +344,7 @@ func TestCacheEmptyFailsOverToMirrorWithoutRefreshCycles(t *testing.T) {
cfg := testConfig(t)
cfg.Upstream.AutoRefresh = true
cfg.Upstream.RefreshRetries = 3
c := New(cfg, nil, nil)
c := New(cfg, nil, nil, nil)
calls := map[string]int{}
c.chatOnceOverride = func(_ context.Context, site config.Site, modelID string, model config.Model,
_ string, _ Route, _ *ServeInfo, _ func(Event) error) error {
@@ -348,7 +374,7 @@ func TestQuotaRotationRecoversWithoutSolve(t *testing.T) {
if _, _, err := solver.ImportCookies("de", "cf_clearance=abc; dsgt_gid="+oldGid, ""); err != nil {
t.Fatalf("seed cookies: %v", err)
}
c := New(cfg, solver, nil)
c := New(cfg, solver, nil, nil)
calls := 0
c.chatOnceOverride = func(_ context.Context, site config.Site, modelID string, model config.Model,
_ string, _ Route, _ *ServeInfo, _ func(Event) error) error {