update 修改
This commit is contained in:
1 parent
fdf77abebc
commit
3a0cd8a867
11 files changed
+674
-305
No files matched your search
+8
-6
@@ -49,7 +49,7 @@
|
||||
- `src/repositories`
|
||||
- SQLite 数据访问
|
||||
- `src/config`
|
||||
- 配置快照管理、Provider 配置规范化、Provider 运行时注册
|
||||
- 配置快照管理、显式 Provider schema / factory、Provider 运行时注册
|
||||
- `src/external`
|
||||
- 上游响应适配、流式探测、Provider 运行时对象
|
||||
- `src/hooks`
|
||||
@@ -78,11 +78,13 @@
|
||||
- 提供原子写回
|
||||
- `get_raw_config()` 返回 `deepcopy`,避免调用方误改内部缓存
|
||||
- `provider_config`
|
||||
- 负责 Provider 配置规范化和校验
|
||||
- 负责 `ProviderConfigSchema`、`RuntimeProviderSpec`、`ProviderRuntimeView`
|
||||
- 负责批量 `build_provider_schemas(...)`、配置规范化和校验
|
||||
- `ProviderManager`
|
||||
- 负责把配置中的 Provider 转成运行时 `LLMProvider`
|
||||
- 维护 `model -> provider` 映射
|
||||
- 维护 Hook 缓存
|
||||
- 只接收已构建好的 `ProviderConfigSchema`
|
||||
- 负责把 schema 转成运行时 `LLMProvider`
|
||||
- 维护 `model -> provider` 映射、只读 `ProviderRuntimeView` 注册表和 Hook 缓存
|
||||
- 对外优先暴露只读接口:`get_provider_view()`、`list_provider_views()`、`list_model_names()`
|
||||
- `LLMProvider`
|
||||
- 运行时 Provider 对象
|
||||
- 当前会复制 `model_list`,避免共享可变引用
|
||||
@@ -94,7 +96,7 @@
|
||||
4. 如果开启白名单,则按 IP 查询用户,并要求 `whitelist_access_enabled=1`
|
||||
5. 校验请求体必须包含 `model`
|
||||
6. 补齐 `stream_options.include_usage=true`
|
||||
7. 用 `ProviderManager.find_provider_by_model(model)` 获取目标 Provider
|
||||
7. 用 `ProviderManager.get_provider_for_model(model)` 获取目标 Provider
|
||||
8. `ProxyService.proxy_request(...)` 转发到上游
|
||||
9. `response_adapter` 处理普通响应或 SSE 响应
|
||||
10. 响应完成后通过回调写入 `request_logs` 并同步更新 `daily_request_stats`
|
||||
|
||||
@@ -1,16 +1,92 @@
|
||||
# LLM_Proxy
|
||||
|
||||
基于 Flask 的 OpenAI 兼容代理服务,聚焦模型转发、访问控制与请求统计。
|
||||
一个基于 Flask + gevent 的 OpenAI 兼容代理服务,重点不只是“填 API Key 转发”,而是通过 Hook 机制把不同上游协议、鉴权方式和请求格式收敛成统一的 OpenAI 风格接口。
|
||||
|
||||
## 功能特性
|
||||
- OpenAI 兼容接口:支持 `POST /v1/chat/completions` 和 `GET /v1/models`
|
||||
- 多 Provider / 多模型路由:按 `model` 自动选择上游 provider
|
||||
- Hook 扩展:单一 `hook` 配置,模块中通过 `Hook(BaseHook)` 按需实现阶段方法
|
||||
- 访问白名单控制:可按客户端 IP 控制是否允许调用代理接口
|
||||
- 管理后台与认证:支持登录、用户管理、统计查询页面与 API
|
||||
- 流式与非流式统一处理:支持 SSE 转发并提取 usage 信息
|
||||
## 项目亮点
|
||||
|
||||
## 使用方式
|
||||
- OpenAI 兼容入口
|
||||
- 提供 `POST /v1/chat/completions` 和 `GET /v1/models`
|
||||
- 多 Provider / 多模型路由
|
||||
- 通过 `provider/model` 形式精确路由到不同上游
|
||||
- Hook 扩展机制
|
||||
- 可在请求头、请求体、响应体三个阶段做定制化改写
|
||||
- 非标准上游适配
|
||||
- 适合处理 Claude / Anthropic 一类需要请求格式转换的场景
|
||||
- 也适合处理经过授权的会话型上游集成,例如额外 Cookie、session token、自定义 header、特定 body 字段
|
||||
- 管理后台
|
||||
- 支持登录、用户管理、Provider 管理、模型探测、白名单控制
|
||||
- 请求统计
|
||||
- 自动记录调用明细和每日聚合 Token 统计
|
||||
|
||||
## 这个项目解决什么问题
|
||||
|
||||
大多数模型服务只需要配置 `api` 和 `api_key` 就能接入,但真实集成里经常会遇到这几类问题:
|
||||
|
||||
- 上游接口不是 OpenAI 格式,需要字段转换
|
||||
- 上游要求额外请求头、Cookie、session token 或特殊参数
|
||||
- 不同厂商响应结构不同,需要统一成 OpenAI 风格
|
||||
- 某些模型接入只在特定客户端或内部工具里可用,需要做受控的协议适配
|
||||
|
||||
LLM_Proxy 的核心价值就在这里:
|
||||
把“代理转发”升级成“可编排、可适配、可管理的统一入口”。
|
||||
|
||||
## 合规说明
|
||||
|
||||
Hook 适配能力的设计目标是支持合法授权前提下的协议兼容和私有集成。
|
||||
|
||||
- 仅应在你拥有访问权限的前提下使用额外的 Cookie、token、header 或会话凭据
|
||||
- 应遵守上游服务的产品条款、访问策略和安全要求
|
||||
- README 和示例不会提供抓包、绕过限制或规避授权的操作步骤
|
||||
|
||||
## 核心能力概览
|
||||
|
||||
### 1. 标准 Provider 接入
|
||||
|
||||
对标准 OpenAI 风格上游,通常只需要配置:
|
||||
|
||||
- `name`
|
||||
- `api`
|
||||
- `api_key`
|
||||
- `model_list`
|
||||
|
||||
这种情况下不需要写 Hook。
|
||||
|
||||
### 2. Hook 方式做协议适配
|
||||
|
||||
对非标准上游,可以通过 Hook 处理:
|
||||
|
||||
- `header_hook`
|
||||
- 注入额外 header、Cookie、鉴权字段
|
||||
- `input_body_hook`
|
||||
- 把 OpenAI 风格请求体转换成上游需要的格式
|
||||
- `output_body_hook`
|
||||
- 把上游响应转换回统一的 OpenAI 风格
|
||||
|
||||
这也是项目当前最有区分度的能力。
|
||||
|
||||
### 3. 管理与运维能力
|
||||
|
||||
- 后台登录
|
||||
- 用户白名单
|
||||
- Provider 配置增删改查
|
||||
- Provider 模型列表探测
|
||||
- 请求日志与聚合统计
|
||||
|
||||
## 架构文档
|
||||
|
||||
完整的 Mermaid 版 4+1 架构视图见:
|
||||
|
||||
- [docs/architecture-4plus1.md](/d:/001Code/008llm/003LLM_Proxy/docs/architecture-4plus1.md)
|
||||
|
||||
其中包含:
|
||||
|
||||
- 逻辑视图
|
||||
- 开发视图
|
||||
- 进程视图
|
||||
- 物理视图
|
||||
- 关键场景时序图
|
||||
|
||||
## 快速开始
|
||||
|
||||
### 1. 安装依赖
|
||||
|
||||
@@ -18,37 +94,34 @@
|
||||
pip install flask gevent requests pyyaml urllib3
|
||||
```
|
||||
|
||||
### 2. 配置服务
|
||||
### 2. 准备配置文件
|
||||
|
||||
编辑 `config.yaml`,至少确认以下字段:
|
||||
复制并编辑 `config.yaml`,至少确认这些字段:
|
||||
|
||||
- `server.host` / `server.port`
|
||||
- `server.host`
|
||||
- `server.port`
|
||||
- `chat.whitelist_enabled`
|
||||
- `providers[].name`
|
||||
- `providers[].api`
|
||||
- `providers[].api_key`
|
||||
- `providers[].model_list`
|
||||
|
||||
可选字段:
|
||||
|
||||
- `admin.username` / `admin.password`
|
||||
- `providers[].api_key`
|
||||
- `providers[].proxy`
|
||||
- `providers[].timeout_seconds`
|
||||
- `providers[].max_retries`
|
||||
- `providers[].verify_ssl`
|
||||
- `providers[].hook`
|
||||
- `logging.path` / `logging.level`
|
||||
- `admin.username`
|
||||
- `admin.password`
|
||||
- `database.path`
|
||||
- `logging.path`
|
||||
- `logging.level`
|
||||
|
||||
`providers[].proxy` 用于为单个 provider 指定上游代理地址,例如:
|
||||
参考样例见:
|
||||
|
||||
```yaml
|
||||
providers:
|
||||
- name: openai
|
||||
api: https://api.openai.com/v1/chat/completions
|
||||
proxy: http://127.0.0.1:7890
|
||||
```
|
||||
|
||||
该代理会同时用于:
|
||||
- 正常聊天转发请求
|
||||
- 管理页中的“拉取模型”请求
|
||||
- [config.sample.yaml](/d:/001Code/008llm/003LLM_Proxy/config.sample.yaml)
|
||||
|
||||
### 3. 启动服务
|
||||
|
||||
@@ -62,51 +135,182 @@ python main.py
|
||||
python main.py --config path/to/config.yaml
|
||||
```
|
||||
|
||||
### 4. 调用代理接口
|
||||
### 4. 调用接口
|
||||
|
||||
获取模型列表:
|
||||
|
||||
```bash
|
||||
curl http://127.0.0.1:22026/v1/models
|
||||
curl http://127.0.0.1:8080/v1/models
|
||||
```
|
||||
|
||||
发起聊天请求:
|
||||
|
||||
```bash
|
||||
curl http://127.0.0.1:22026/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "your-model-name",
|
||||
"messages": [{"role": "user", "content": "hello"}],
|
||||
"stream": false
|
||||
}'
|
||||
curl http://127.0.0.1:8080/v1/chat/completions ^
|
||||
-H "Content-Type: application/json" ^
|
||||
-d "{\"model\":\"volc/glm-4-7-251222\",\"messages\":[{\"role\":\"user\",\"content\":\"hello\"}],\"stream\":false}"
|
||||
```
|
||||
|
||||
## Hook API(类接口)
|
||||
## 配置示例
|
||||
|
||||
Provider 配置使用单一 `hook` 字段。
|
||||
### 标准 OpenAI 风格 Provider
|
||||
|
||||
```yaml
|
||||
providers:
|
||||
- name: your-provider
|
||||
api: https://example.com/v1/chat/completions
|
||||
api_key: your-key
|
||||
- name: openai
|
||||
api: https://api.openai.com/v1/chat/completions
|
||||
api_key: ${OPENAI_API_KEY}
|
||||
model_list:
|
||||
- your-model
|
||||
hook: example_hook.py
|
||||
- gpt-4.1
|
||||
- gpt-4o-mini
|
||||
```
|
||||
|
||||
Hook 模块必须导出一个名为 `Hook` 的类,通常继承 `BaseHook`:
|
||||
### 带 Hook 的 Provider
|
||||
|
||||
```yaml
|
||||
providers:
|
||||
- name: anthropic
|
||||
api: https://example.com/v1/chat/completions
|
||||
api_key:
|
||||
model_list:
|
||||
- claude-sonnet
|
||||
hook: anthropic_compat.py
|
||||
```
|
||||
|
||||
这类 Hook 常见用途:
|
||||
|
||||
- 把 OpenAI 风格 `messages` 转成上游要求的结构
|
||||
- 注入额外鉴权 header / Cookie
|
||||
- 修正响应字段,统一 `choices`、`usage`、`model`
|
||||
|
||||
## Hook API
|
||||
|
||||
Hook 模块需要导出一个名为 `Hook` 的类,通常继承:
|
||||
|
||||
```python
|
||||
from src.hooks import BaseHook, HookContext
|
||||
|
||||
class Hook(BaseHook):
|
||||
def header_hook(self, ctx: HookContext, headers: dict[str, str]) -> dict[str, str]:
|
||||
return headers
|
||||
```
|
||||
|
||||
可选实现的方法:
|
||||
|
||||
- `header_hook(ctx, headers) -> headers`
|
||||
- `input_body_hook(ctx, body) -> body`
|
||||
- `output_body_hook(ctx, body) -> body`
|
||||
|
||||
示例:
|
||||
|
||||
```python
|
||||
from typing import Any
|
||||
|
||||
from src.hooks import BaseHook, HookContext
|
||||
|
||||
|
||||
class Hook(BaseHook):
|
||||
"""Anthropic / 私有网关适配示例骨架。"""
|
||||
|
||||
def header_hook(self, ctx: HookContext, headers: dict[str, str]) -> dict[str, str]:
|
||||
headers["X-Client"] = "llm-proxy"
|
||||
return headers
|
||||
|
||||
def input_body_hook(self, ctx: HookContext, body: dict[str, Any]) -> dict[str, Any]:
|
||||
return body
|
||||
|
||||
def output_body_hook(self, ctx: HookContext, body: Any) -> Any:
|
||||
return body
|
||||
```
|
||||
|
||||
现成示例见:
|
||||
|
||||
- [hooks/example_hook.py](/d:/001Code/008llm/003LLM_Proxy/hooks/example_hook.py)
|
||||
|
||||
## Hook 适配的典型场景
|
||||
|
||||
### Claude / Anthropic 格式转换
|
||||
|
||||
某些上游并不直接接受 OpenAI 风格的:
|
||||
|
||||
- `messages`
|
||||
- `tools`
|
||||
- `stream` / `usage`
|
||||
|
||||
这时可以在 Hook 中做双向转换:
|
||||
|
||||
- 请求进入代理时,从 OpenAI 格式改写成上游格式
|
||||
- 响应返回客户端前,再还原成统一的 OpenAI 风格
|
||||
|
||||
### 会话型或私有上游集成
|
||||
|
||||
某些经过授权的私有集成可能要求:
|
||||
|
||||
- 额外 Cookie
|
||||
- session token
|
||||
- 自定义 header
|
||||
- 特定 body 标记
|
||||
|
||||
这时可以通过 `header_hook` 和 `input_body_hook` 注入这些字段,实现受控反代接入。
|
||||
|
||||
### 响应清洗或增强
|
||||
|
||||
有些上游:
|
||||
|
||||
- 不返回标准 `usage`
|
||||
- 返回字段名不一致
|
||||
- 流式块格式不完全兼容
|
||||
|
||||
可以在 `output_body_hook` 中统一处理。
|
||||
|
||||
## 后台能力
|
||||
|
||||
后台当前提供:
|
||||
|
||||
- 登录认证
|
||||
- 用户管理
|
||||
- 白名单开关
|
||||
- Provider 管理
|
||||
- 模型探测
|
||||
- 请求日志
|
||||
- 聚合统计
|
||||
|
||||
主要页面:
|
||||
|
||||
- `/`
|
||||
- `/users`
|
||||
- `/providers`
|
||||
|
||||
## 数据存储
|
||||
|
||||
默认使用 SQLite:
|
||||
|
||||
- 数据库文件:`data/requests.db`
|
||||
- 配置文件:`config.yaml`
|
||||
- 日志目录:`logs/`
|
||||
- Hook 目录:`hooks/`
|
||||
|
||||
主要表:
|
||||
|
||||
- `users`
|
||||
- `request_logs`
|
||||
- `daily_request_stats`
|
||||
|
||||
## 当前架构判断
|
||||
|
||||
这个项目当前最值得保留的设计不是“又一个 OpenAI 代理”,而是:
|
||||
|
||||
**把 Provider 转发、协议适配、会话注入和后台管理放进了同一个可控边界里。**
|
||||
|
||||
也就是说:
|
||||
|
||||
- 标准 Provider 可以零适配接入
|
||||
- 非标准 Provider 可以最小成本做 Hook 兼容
|
||||
- 业务方看到的仍然是统一的 OpenAI 接口
|
||||
|
||||
## 后续建议
|
||||
|
||||
如果后续继续演进,优先建议:
|
||||
|
||||
- 增加更多官方示例 Hook
|
||||
- 例如 Claude 兼容骨架
|
||||
- 例如 session header 注入骨架
|
||||
- 把 Provider 配置 schema 进一步显式化
|
||||
- 增加自动化测试覆盖 Hook 输入输出转换
|
||||
- 增加 README 中的真实接入案例文档
|
||||
+159
-97
@@ -6,62 +6,74 @@
|
||||
- 涉及模块职责、依赖关系、主请求链路、运行时状态、配置加载/重载机制、Hook 机制、部署拓扑的修改时,需要同步更新本文件。
|
||||
- 如果代码实现与本图不一致,以代码为准,但提交中应补齐文档更新。
|
||||
|
||||
## 0. 架构摘要
|
||||
## 0. Architecture Summary
|
||||
|
||||
系统当前是一个单进程、分层式单体应用,按职责分成两条主轴:
|
||||
- 控制平面:后台管理、认证、用户、Provider 配置、系统设置。
|
||||
- 数据平面:OpenAI 兼容代理、模型路由、上游请求转发、响应适配、日志落库。
|
||||
|
||||
- 控制平面:后台管理、认证、用户、Provider 配置、系统设置
|
||||
- 数据平面:OpenAI 兼容代理、模型路由、上游请求转发、响应适配、日志落库
|
||||
|
||||
这个项目当前最有区分度的能力不是“普通 API 代理”,而是:
|
||||
|
||||
- 通过 Hook 机制在请求头、请求体、响应体三个阶段做协议适配
|
||||
- 既能接标准 OpenAI 风格上游,也能接非标准协议上游
|
||||
- 适合处理 Claude / Anthropic 一类需要格式转换的接入场景
|
||||
- 也适合处理经过授权的会话型集成,例如额外 Cookie、session token、自定义 header 或私有参数
|
||||
|
||||
这里强调的是“授权前提下的兼容与适配”,不是绕过访问控制。
|
||||
|
||||
关键代码位置:
|
||||
|
||||
- 启动与装配:[main.py](/d:/001Code/008llm/003LLM_Proxy/main.py)
|
||||
- 组合根:[application.py](/d:/001Code/008llm/003LLM_Proxy/src/application/application.py)
|
||||
- 配置与 Provider 运行时:[config_manager.py](/d:/001Code/008llm/003LLM_Proxy/src/config/config_manager.py) [provider_manager.py](/d:/001Code/008llm/003LLM_Proxy/src/config/provider_manager.py)
|
||||
- Hook 协议:[contracts.py](/d:/001Code/008llm/003LLM_Proxy/src/hooks/contracts.py)
|
||||
- 代理主链路:[proxy_controller.py](/d:/001Code/008llm/003LLM_Proxy/src/presentation/proxy_controller.py) [proxy_service.py](/d:/001Code/008llm/003LLM_Proxy/src/services/proxy_service.py)
|
||||
|
||||
## 1. Logical View
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Browser[浏览器管理端]
|
||||
ApiClient[OpenAI 兼容 API Client]
|
||||
Browser["浏览器管理端"]
|
||||
ApiClient["OpenAI 兼容 API Client"]
|
||||
|
||||
subgraph App["LLM_Proxy 应用"]
|
||||
subgraph Presentation["Presentation"]
|
||||
WebCtl[WebController]
|
||||
AuthCtl[AuthenticationController]
|
||||
UserCtl[UserController]
|
||||
ProviderCtl[ProviderController]
|
||||
ProxyCtl[ProxyController]
|
||||
WebCtl["WebController"]
|
||||
AuthCtl["AuthenticationController"]
|
||||
UserCtl["UserController"]
|
||||
ProviderCtl["ProviderController"]
|
||||
ProxyCtl["ProxyController"]
|
||||
end
|
||||
|
||||
subgraph Services["Services"]
|
||||
AuthSvc[AuthenticationService]
|
||||
UserSvc[UserService]
|
||||
ProviderSvc[ProviderService]
|
||||
DiscoverySvc[ModelDiscoveryService]
|
||||
SettingsSvc[SettingsService]
|
||||
ProxySvc[ProxyService]
|
||||
LogSvc[LogService]
|
||||
AuthSvc["AuthenticationService"]
|
||||
UserSvc["UserService"]
|
||||
ProviderSvc["ProviderService"]
|
||||
DiscoverySvc["ModelDiscoveryService"]
|
||||
SettingsSvc["SettingsService"]
|
||||
ProxySvc["ProxyService"]
|
||||
LogSvc["LogService"]
|
||||
end
|
||||
|
||||
subgraph ConfigRuntime["Config / Provider Runtime"]
|
||||
ConfigMgr[ConfigManager]
|
||||
ProviderCfg[provider_config]
|
||||
ProviderMgr[ProviderManager]
|
||||
LLMProvider[LLMProvider]
|
||||
ConfigMgr["ConfigManager"]
|
||||
ProviderCfg["provider_config"]
|
||||
ProviderMgr["ProviderManager"]
|
||||
LLMProvider["LLMProvider"]
|
||||
end
|
||||
|
||||
subgraph Persistence["Persistence"]
|
||||
UserRepo[UserRepository]
|
||||
LogRepo[LogRepository]
|
||||
SQLite[(SQLite)]
|
||||
UserRepo["UserRepository"]
|
||||
LogRepo["LogRepository"]
|
||||
SQLite[("SQLite")]
|
||||
end
|
||||
|
||||
subgraph Integration["External Integration"]
|
||||
Hooks[Hook Modules]
|
||||
Adapter[response_adapter]
|
||||
Probe[stream_probe]
|
||||
Upstream[Upstream LLM Providers]
|
||||
Hooks["Hook Modules"]
|
||||
Adapter["response_adapter"]
|
||||
Probe["stream_probe"]
|
||||
Upstream["Upstream LLM Providers"]
|
||||
end
|
||||
end
|
||||
|
||||
@@ -99,21 +111,30 @@ flowchart LR
|
||||
LogSvc --> LogRepo
|
||||
UserRepo --> SQLite
|
||||
LogRepo --> SQLite
|
||||
ConfigMgr --> ConfigFile[(config.yaml)]
|
||||
ConfigMgr --> ConfigFile[("config.yaml")]
|
||||
```
|
||||
|
||||
逻辑划分:
|
||||
- `Presentation` 负责 HTTP 路由、鉴权入口、页面渲染和请求/响应封装。
|
||||
- `Services` 负责用例编排,不直接持有全局可变配置。
|
||||
- `Config / Provider Runtime` 负责 YAML 配置快照、Provider 规范化、模型到 Provider 的运行时映射。
|
||||
- `Persistence` 负责 SQLite 读写。
|
||||
- `External Integration` 负责上游协议适配、流式探测、Hook 扩展和上游 Provider 集成。
|
||||
|
||||
- `Presentation` 负责 HTTP 路由、鉴权入口、页面渲染和请求/响应封装
|
||||
- `Services` 负责用例编排,不直接共享全局可变配置
|
||||
- `Config / Provider Runtime` 负责 YAML 配置快照、显式 `ProviderConfigSchema` / `RuntimeProviderSpec` 工厂、只读运行时视图和模型到 Provider 的运行时映射
|
||||
- `Persistence` 负责 SQLite 读写
|
||||
- `External Integration` 负责上游协议适配、流式探测、Hook 扩展和上游 Provider 集成
|
||||
|
||||
其中最重要的扩展边界是 Hook:
|
||||
|
||||
- `header_hook`
|
||||
- `input_body_hook`
|
||||
- `output_body_hook`
|
||||
|
||||
它使系统从“静态代理”升级成“可编排协议适配器”。
|
||||
|
||||
## 2. Development View
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
Main[main.py]
|
||||
Main["main.py"]
|
||||
|
||||
subgraph Src["src/"]
|
||||
Application["application/"]
|
||||
@@ -155,37 +176,39 @@ flowchart TB
|
||||
```
|
||||
|
||||
开发视图解读:
|
||||
- `application` 是组合根,不承载具体业务规则。
|
||||
- `presentation -> services` 是主要调用方向。
|
||||
- `services -> repositories/config/external` 是业务侧依赖方向。
|
||||
- `config` 既不是传统 repository,也不是 service,更像运行时配置与 Provider 注册子系统。
|
||||
|
||||
- `application` 是组合根,不承载具体业务规则
|
||||
- `presentation -> services` 是主要调用方向
|
||||
- `services -> repositories/config/external` 是业务侧依赖方向
|
||||
- `config` 更像运行时配置和 Provider 注册子系统
|
||||
- `hooks` 是面向定制集成的外部扩展点
|
||||
|
||||
## 3. Process View
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph Process["单个 Python 进程"]
|
||||
WSGI[gevent WSGIServer]
|
||||
FlaskApp[Flask App]
|
||||
WSGI["gevent WSGIServer"]
|
||||
FlaskApp["Flask App"]
|
||||
|
||||
subgraph RuntimeState["进程内运行时状态"]
|
||||
SessionState[AuthenticationService\n内存 Session]
|
||||
UserCache[UserService\nIP -> User 缓存]
|
||||
ConfigCache[ConfigManager\n配置快照缓存]
|
||||
ProviderRegistry[ProviderManager\nmodel -> provider 映射\nhook cache]
|
||||
HttpLocal[ProxyService\nthread-local requests.Session]
|
||||
SessionState["AuthenticationService\n内存 Session"]
|
||||
UserCache["UserService\nIP -> User 缓存"]
|
||||
ConfigCache["ConfigManager\n配置快照缓存"]
|
||||
ProviderRegistry["ProviderManager\nmodel -> provider 映射\nhook cache"]
|
||||
HttpLocal["ProxyService\nthread-local requests.Session"]
|
||||
end
|
||||
end
|
||||
|
||||
subgraph Local["本地资源"]
|
||||
ConfigFile[(config.yaml)]
|
||||
DB[(data/requests.db)]
|
||||
HooksDir[(hooks/*.py)]
|
||||
LogFiles[(logs/*.log)]
|
||||
ConfigFile[("config.yaml")]
|
||||
DB[("data/requests.db")]
|
||||
HooksDir[("hooks/*.py")]
|
||||
LogFiles[("logs/*.log")]
|
||||
end
|
||||
|
||||
subgraph Remote["外部网络资源"]
|
||||
Upstream[Upstream LLM Providers]
|
||||
Upstream["Upstream LLM Providers"]
|
||||
end
|
||||
|
||||
WSGI --> FlaskApp
|
||||
@@ -203,58 +226,61 @@ flowchart TB
|
||||
```
|
||||
|
||||
进程视图要点:
|
||||
- 当前是单进程单实例内存态设计。
|
||||
- `AuthenticationService` 的 Session 存在进程内存中,重启后失效。
|
||||
- `UserService` 有 IP 维度缓存。
|
||||
- `ProviderManager` 持有运行时模型路由表和 Hook 缓存。
|
||||
- `ProxyService` 维护 thread-local `requests.Session`。
|
||||
- 如果未来引入多实例部署,这些内存态要重新设计。
|
||||
|
||||
- 当前是单进程单实例内存态设计
|
||||
- `AuthenticationService` 的 Session 存在进程内存中,重启后失效
|
||||
- `UserService` 有 IP 维度缓存
|
||||
- `ProviderManager` 持有运行时模型路由表、只读 `ProviderRuntimeView` 注册表和 Hook 缓存
|
||||
- `ProxyService` 维护 thread-local `requests.Session`
|
||||
- Hook 模块以运行时动态加载方式参与请求路径
|
||||
- 如果未来引入多实例部署,这些内存态要重新设计
|
||||
|
||||
## 4. Physical View
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph Clients["客户端"]
|
||||
Browser[浏览器管理端]
|
||||
SDK[SDK / Curl / OpenAI Client]
|
||||
Browser["浏览器管理端"]
|
||||
SDK["SDK / Curl / OpenAI Client"]
|
||||
end
|
||||
|
||||
subgraph Host["部署主机"]
|
||||
subgraph Service["LLM_Proxy 服务进程"]
|
||||
Entry[main.py]
|
||||
Server[gevent WSGIServer]
|
||||
App[Flask + Controllers + Services]
|
||||
Entry["main.py"]
|
||||
Server["gevent WSGIServer"]
|
||||
App["Flask + Controllers + Services"]
|
||||
end
|
||||
|
||||
ConfigFile[(config.yaml)]
|
||||
DB[(data/requests.db)]
|
||||
HooksDir[(hooks/*.py)]
|
||||
Logs[(logs/*.log)]
|
||||
ConfigFile[("config.yaml")]
|
||||
DB[("data/requests.db")]
|
||||
HooksDir[("hooks/*.py")]
|
||||
Logs[("logs/*.log")]
|
||||
end
|
||||
|
||||
subgraph Providers["外部 Provider"]
|
||||
P1[Provider A]
|
||||
P2[Provider B]
|
||||
PN[Provider N]
|
||||
P1["Provider A"]
|
||||
P2["Provider B"]
|
||||
PN["Provider N"]
|
||||
end
|
||||
|
||||
Browser <--HTTP/HTTPS--> Server
|
||||
SDK <--HTTP/HTTPS--> Server
|
||||
Browser <-->|HTTP/HTTPS| Server
|
||||
SDK <-->|HTTP/HTTPS| Server
|
||||
|
||||
App --> ConfigFile
|
||||
App --> DB
|
||||
App --> HooksDir
|
||||
App --> Logs
|
||||
|
||||
App <--HTTPS--> P1
|
||||
App <--HTTPS--> P2
|
||||
App <--HTTPS--> PN
|
||||
App <-->|HTTPS| P1
|
||||
App <-->|HTTPS| P2
|
||||
App <-->|HTTPS| PN
|
||||
```
|
||||
|
||||
物理视图要点:
|
||||
- 部署结构目前非常简单,单机即可运行。
|
||||
- 本地状态包括配置文件、SQLite、日志文件、Hook 文件。
|
||||
- 外部依赖主要是上游模型 Provider 的 HTTP API。
|
||||
|
||||
- 当前部署结构很简单,单机即可运行
|
||||
- 本地状态包括配置文件、SQLite、日志文件、Hook 文件
|
||||
- 外部依赖主要是上游模型 Provider 的 HTTP API
|
||||
|
||||
## 5. Scenario View
|
||||
|
||||
@@ -299,16 +325,17 @@ sequenceDiagram
|
||||
ProviderCtl->>ProviderSvc: update_provider(payload)
|
||||
ProviderSvc->>ConfigMgr: get_raw_config()
|
||||
ConfigMgr-->>ProviderSvc: config snapshot
|
||||
ProviderSvc->>ProviderCfg: normalize_provider_payload()
|
||||
ProviderSvc->>ProviderCfg: ProviderConfigSchema.from_payload()
|
||||
ProviderSvc->>ProviderCfg: validate_provider_definitions()
|
||||
ProviderSvc->>ConfigMgr: write_raw_config(updated config)
|
||||
ProviderSvc->>App: reload_providers()
|
||||
App->>ConfigMgr: reload()
|
||||
App->>ConfigMgr: get_raw_config()
|
||||
App->>ProviderMgr: load_providers(providers_config)
|
||||
ProviderMgr->>ProviderCfg: normalize_runtime_provider_config()
|
||||
App->>ProviderCfg: build_provider_schemas(providers_config)
|
||||
App->>ProviderMgr: load_providers(provider_schemas)
|
||||
ProviderMgr->>ProviderCfg: RuntimeProviderSpec.from_schema()
|
||||
ProviderMgr->>Hooks: load hook modules if configured
|
||||
ProviderMgr-->>App: provider registry refreshed
|
||||
ProviderMgr-->>App: runtime registry + readonly views refreshed
|
||||
ProviderSvc-->>ProviderCtl: updated provider
|
||||
ProviderCtl-->>Admin: 200 OK
|
||||
```
|
||||
@@ -354,7 +381,7 @@ sequenceDiagram
|
||||
UserSvc-->>ProxyCtl: user / None
|
||||
end
|
||||
|
||||
ProxyCtl->>ProviderMgr: find_provider_by_model(model)
|
||||
ProxyCtl->>ProviderMgr: get_provider_for_model(model)
|
||||
ProviderMgr-->>ProxyCtl: LLMProvider
|
||||
ProxyCtl->>ProxySvc: proxy_request(provider, body, headers, on_complete)
|
||||
ProxySvc->>Hook: header_hook / input_body_hook
|
||||
@@ -367,24 +394,59 @@ sequenceDiagram
|
||||
LogSvc->>LogRepo: insert(...)
|
||||
```
|
||||
|
||||
## 6. 当前架构判断
|
||||
### 5.5 非标准上游协议适配
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
actor Client as OpenAI Client
|
||||
participant ProxyCtl as ProxyController
|
||||
participant ProviderMgr as ProviderManager
|
||||
participant ProxySvc as ProxyService
|
||||
participant Hook as Compatibility Hook
|
||||
participant Upstream as Non-OpenAI Upstream
|
||||
participant Adapter as response_adapter
|
||||
|
||||
Client->>ProxyCtl: POST /v1/chat/completions (OpenAI format)
|
||||
ProxyCtl->>ProviderMgr: get_provider_for_model(model)
|
||||
ProviderMgr-->>ProxyCtl: runtime provider with hook
|
||||
ProxyCtl->>ProxySvc: proxy_request(...)
|
||||
ProxySvc->>Hook: header_hook(headers)
|
||||
Note over Hook: 注入额外 header / Cookie / session token\n仅用于合法授权的集成场景
|
||||
ProxySvc->>Hook: input_body_hook(body)
|
||||
Note over Hook: 把 OpenAI body 改写成 Anthropic / 私有上游格式
|
||||
ProxySvc->>Upstream: POST adapted request
|
||||
Upstream-->>ProxySvc: non-standard response
|
||||
ProxySvc->>Adapter: build_proxy_response(...)
|
||||
Adapter->>Hook: output_body_hook(body)
|
||||
Note over Hook: 统一响应结构、usage、choices、stream chunk
|
||||
Adapter-->>Client: OpenAI-compatible response
|
||||
```
|
||||
|
||||
## 6. Current Architectural Assessment
|
||||
|
||||
当前架构优点:
|
||||
- 模块划分清楚,学习成本低。
|
||||
- 单进程单文件配置的运维复杂度很低。
|
||||
- 控制平面和数据平面在职责上已经开始分离。
|
||||
- Hook 扩展点足够轻量,适合小规模快速演进。
|
||||
|
||||
- 模块划分清楚,学习成本低
|
||||
- 单进程单文件配置的运维复杂度很低
|
||||
- 控制平面和数据平面在职责上已经分离出清晰轮廓
|
||||
- Hook 扩展点足够轻量,但表达力很强
|
||||
- 兼容层能力使项目不局限于“标准 API Key 型接入”
|
||||
|
||||
当前架构边界:
|
||||
- 运行时状态仍集中在单进程内存中,不适合多实例横向扩展。
|
||||
- `presentation` 层仍承担部分接口协议细节和错误映射责任。
|
||||
- `config` 子系统目前同时承担配置快照、配置规范化、Provider 注册三类职责。
|
||||
- Hook 是运行时动态加载,扩展灵活,但也引入了热更新和可观测性上的复杂性。
|
||||
|
||||
## 7. 后续演进建议
|
||||
- 运行时状态仍集中在单进程内存中,不适合多实例横向扩展
|
||||
- `presentation` 层仍承担部分接口协议细节和错误映射责任
|
||||
- `config` 子系统已经收敛为配置快照、显式 schema/factory、Provider 注册三类职责,但仍集中在同一 package 中
|
||||
- Hook 动态加载带来了灵活性,也带来了调试、观测和测试复杂性
|
||||
|
||||
如果后面继续演进,优先级建议是:
|
||||
- 把 `provider_config` 进一步收敛成显式 schema / factory。
|
||||
- 为 `ProviderManager` 增加更清晰的只读运行时接口。
|
||||
- 评估是否把 Session、Provider Registry、User IP 缓存从单进程内存态解耦。
|
||||
- 补一份 ADR,说明为什么当前仍选择单体 + 单进程 + SQLite。
|
||||
## 7. Evolution Suggestions
|
||||
|
||||
如果后面继续演进,优先建议是:
|
||||
|
||||
- 增加官方维护的 Hook 示例库
|
||||
- Claude / Anthropic 兼容示例
|
||||
- session header / Cookie 注入示例
|
||||
- 私有网关适配示例
|
||||
- 基于 `ProviderRuntimeView` 增加后台运行时诊断或只读观测能力
|
||||
- 补充 Hook 输入输出转换测试
|
||||
- 评估是否把 Session、Provider Registry、User IP 缓存从单进程内存态解耦
|
||||
@@ -8,7 +8,7 @@ from pathlib import Path
|
||||
from flask import request
|
||||
|
||||
from .app_context import AppContext
|
||||
from ..config import ConfigManager, ProviderManager
|
||||
from ..config import ConfigManager, ProviderManager, build_provider_schemas
|
||||
from ..presentation import (
|
||||
AuthenticationController,
|
||||
ProviderController,
|
||||
@@ -187,8 +187,11 @@ class Application:
|
||||
def reload_providers(self) -> None:
|
||||
self._config_manager.reload()
|
||||
config_dict = self._config_manager.get_raw_config()
|
||||
providers_config = config_dict.get('providers', [])
|
||||
self._provider_manager.load_providers(providers_config)
|
||||
providers_config = config_dict.get("providers", [])
|
||||
if providers_config is None:
|
||||
providers_config = []
|
||||
provider_schemas = build_provider_schemas(providers_config)
|
||||
self._provider_manager.load_providers(provider_schemas)
|
||||
|
||||
def run(self) -> None:
|
||||
"""启动 WSGI 服务。"""
|
||||
|
||||
@@ -3,9 +3,19 @@
|
||||
"""配置层导出。"""
|
||||
|
||||
from .config_manager import ConfigManager
|
||||
from .provider_config import (
|
||||
ProviderConfigSchema,
|
||||
ProviderRuntimeView,
|
||||
RuntimeProviderSpec,
|
||||
build_provider_schemas,
|
||||
)
|
||||
from .provider_manager import ProviderManager
|
||||
|
||||
__all__ = [
|
||||
"ConfigManager",
|
||||
"ProviderConfigSchema",
|
||||
"RuntimeProviderSpec",
|
||||
"ProviderRuntimeView",
|
||||
"build_provider_schemas",
|
||||
"ProviderManager",
|
||||
]
|
||||
+147
-72
@@ -1,9 +1,10 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Provider 配置的规范化与校验。"""
|
||||
"""Provider 配置 schema、factory 与校验。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, Dict, List, Mapping, Optional, Sequence
|
||||
|
||||
from ..utils.net import normalize_proxy_url
|
||||
@@ -90,95 +91,169 @@ def normalize_model_list(value: Any) -> List[str]:
|
||||
return models
|
||||
|
||||
|
||||
def normalize_provider_payload(payload: Mapping[str, Any]) -> Dict[str, Any]:
|
||||
if not isinstance(payload, Mapping):
|
||||
raise ValueError("Provider payload must be an object")
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ProviderConfigSchema:
|
||||
"""显式表示配置文件中的单个 provider。"""
|
||||
|
||||
name = clean_optional_string(payload.get("name"))
|
||||
api = clean_optional_string(payload.get("api"))
|
||||
if name is None:
|
||||
raise ValueError("Provider name is required")
|
||||
if api is None:
|
||||
raise ValueError("Provider api is required")
|
||||
name: str
|
||||
api: str
|
||||
api_key: Optional[str] = None
|
||||
proxy: Optional[str] = None
|
||||
timeout_seconds: Optional[int] = None
|
||||
max_retries: Optional[int] = None
|
||||
verify_ssl: Optional[bool] = None
|
||||
model_list: tuple[str, ...] = ()
|
||||
hook: Optional[str] = None
|
||||
|
||||
normalized: Dict[str, Any] = {
|
||||
"name": name,
|
||||
"api": api,
|
||||
}
|
||||
@classmethod
|
||||
def from_payload(cls, payload: Mapping[str, Any]) -> "ProviderConfigSchema":
|
||||
"""从管理接口 payload 构造规范化 schema。"""
|
||||
return cls._from_mapping(payload)
|
||||
|
||||
api_key = clean_optional_string(payload.get("api_key"))
|
||||
if api_key is not None:
|
||||
normalized["api_key"] = api_key
|
||||
@classmethod
|
||||
def from_mapping(cls, config: Mapping[str, Any]) -> "ProviderConfigSchema":
|
||||
"""从配置文件对象构造规范化 schema。"""
|
||||
return cls._from_mapping(config)
|
||||
|
||||
proxy = normalize_proxy_url(payload.get("proxy"))
|
||||
if proxy is not None:
|
||||
normalized["proxy"] = proxy
|
||||
@classmethod
|
||||
def _from_mapping(cls, config: Mapping[str, Any]) -> "ProviderConfigSchema":
|
||||
if not isinstance(config, Mapping):
|
||||
raise ValueError("Provider config must be an object")
|
||||
|
||||
timeout_seconds = parse_optional_positive_int(payload.get("timeout_seconds"))
|
||||
if timeout_seconds is not None:
|
||||
normalized["timeout_seconds"] = timeout_seconds
|
||||
name = clean_optional_string(config.get("name"))
|
||||
api = clean_optional_string(config.get("api"))
|
||||
if name is None:
|
||||
raise ValueError("Provider name is required")
|
||||
if api is None:
|
||||
raise ValueError("Provider api is required")
|
||||
|
||||
max_retries = parse_optional_positive_int(payload.get("max_retries"))
|
||||
if max_retries is not None:
|
||||
normalized["max_retries"] = max_retries
|
||||
return cls(
|
||||
name=name,
|
||||
api=api,
|
||||
api_key=clean_optional_string(config.get("api_key")),
|
||||
proxy=normalize_proxy_url(config.get("proxy")),
|
||||
timeout_seconds=parse_optional_positive_int(config.get("timeout_seconds")),
|
||||
max_retries=parse_optional_positive_int(config.get("max_retries")),
|
||||
verify_ssl=parse_optional_bool(config.get("verify_ssl")),
|
||||
model_list=tuple(normalize_model_list(config.get("model_list"))),
|
||||
hook=clean_optional_string(config.get("hook")),
|
||||
)
|
||||
|
||||
verify_ssl = parse_optional_bool(payload.get("verify_ssl"))
|
||||
if verify_ssl is not None:
|
||||
normalized["verify_ssl"] = verify_ssl
|
||||
def to_mapping(self) -> Dict[str, Any]:
|
||||
"""转换为适合写回配置文件的普通 dict。"""
|
||||
config: Dict[str, Any] = {
|
||||
"name": self.name,
|
||||
"api": self.api,
|
||||
}
|
||||
|
||||
model_list = normalize_model_list(payload.get("model_list"))
|
||||
if model_list:
|
||||
normalized["model_list"] = model_list
|
||||
if self.api_key is not None:
|
||||
config["api_key"] = self.api_key
|
||||
if self.proxy is not None:
|
||||
config["proxy"] = self.proxy
|
||||
if self.timeout_seconds is not None:
|
||||
config["timeout_seconds"] = self.timeout_seconds
|
||||
if self.max_retries is not None:
|
||||
config["max_retries"] = self.max_retries
|
||||
if self.verify_ssl is not None:
|
||||
config["verify_ssl"] = self.verify_ssl
|
||||
if self.model_list:
|
||||
config["model_list"] = list(self.model_list)
|
||||
if self.hook is not None:
|
||||
config["hook"] = self.hook
|
||||
|
||||
hook = clean_optional_string(payload.get("hook"))
|
||||
if hook is not None:
|
||||
normalized["hook"] = hook
|
||||
|
||||
return normalized
|
||||
return config
|
||||
|
||||
|
||||
def normalize_runtime_provider_config(config: Mapping[str, Any]) -> Dict[str, Any]:
|
||||
if not isinstance(config, Mapping):
|
||||
raise ValueError("Provider config must be an object")
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RuntimeProviderSpec:
|
||||
"""显式表示运行时使用的 provider 规格。"""
|
||||
|
||||
name = clean_optional_string(config.get("name"))
|
||||
api = clean_optional_string(config.get("api"))
|
||||
if name is None:
|
||||
raise ValueError("Provider name is required")
|
||||
if api is None:
|
||||
raise ValueError("Provider api is required")
|
||||
name: str
|
||||
api: str
|
||||
api_key: str
|
||||
model_list: tuple[str, ...]
|
||||
proxy: Optional[str]
|
||||
timeout_seconds: int
|
||||
max_retries: int
|
||||
verify_ssl: bool
|
||||
hook: Optional[str]
|
||||
|
||||
return {
|
||||
"name": name,
|
||||
"api": api,
|
||||
"api_key": clean_optional_string(config.get("api_key")) or "",
|
||||
"model_list": normalize_model_list(config.get("model_list")),
|
||||
"proxy": normalize_proxy_url(config.get("proxy")),
|
||||
"timeout_seconds": parse_optional_positive_int(
|
||||
config.get("timeout_seconds"),
|
||||
default=DEFAULT_PROVIDER_TIMEOUT_SECONDS,
|
||||
),
|
||||
"max_retries": parse_optional_positive_int(
|
||||
config.get("max_retries"),
|
||||
default=DEFAULT_PROVIDER_MAX_RETRIES,
|
||||
),
|
||||
"verify_ssl": parse_optional_bool(
|
||||
config.get("verify_ssl"),
|
||||
default=DEFAULT_PROVIDER_VERIFY_SSL,
|
||||
),
|
||||
"hook": clean_optional_string(config.get("hook")),
|
||||
}
|
||||
@classmethod
|
||||
def from_mapping(cls, config: Mapping[str, Any]) -> "RuntimeProviderSpec":
|
||||
return cls.from_schema(ProviderConfigSchema.from_mapping(config))
|
||||
|
||||
@classmethod
|
||||
def from_schema(cls, config: ProviderConfigSchema) -> "RuntimeProviderSpec":
|
||||
return cls(
|
||||
name=config.name,
|
||||
api=config.api,
|
||||
api_key=config.api_key or "",
|
||||
model_list=config.model_list,
|
||||
proxy=config.proxy,
|
||||
timeout_seconds=config.timeout_seconds or DEFAULT_PROVIDER_TIMEOUT_SECONDS,
|
||||
max_retries=config.max_retries or DEFAULT_PROVIDER_MAX_RETRIES,
|
||||
verify_ssl=(
|
||||
config.verify_ssl
|
||||
if config.verify_ssl is not None
|
||||
else DEFAULT_PROVIDER_VERIFY_SSL
|
||||
),
|
||||
hook=config.hook,
|
||||
)
|
||||
|
||||
|
||||
def validate_provider_definitions(providers: Sequence[Mapping[str, Any]]) -> None:
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ProviderRuntimeView:
|
||||
"""ProviderManager 暴露的只读运行时视图。"""
|
||||
|
||||
name: str
|
||||
api: str
|
||||
model_list: tuple[str, ...]
|
||||
proxy: Optional[str]
|
||||
timeout_seconds: int
|
||||
max_retries: int
|
||||
verify_ssl: bool
|
||||
hook: Optional[str]
|
||||
|
||||
@classmethod
|
||||
def from_spec(cls, spec: RuntimeProviderSpec) -> "ProviderRuntimeView":
|
||||
return cls(
|
||||
name=spec.name,
|
||||
api=spec.api,
|
||||
model_list=spec.model_list,
|
||||
proxy=spec.proxy,
|
||||
timeout_seconds=spec.timeout_seconds,
|
||||
max_retries=spec.max_retries,
|
||||
verify_ssl=spec.verify_ssl,
|
||||
hook=spec.hook,
|
||||
)
|
||||
|
||||
|
||||
def build_provider_schemas(
|
||||
providers: Sequence[Mapping[str, Any]],
|
||||
) -> tuple[ProviderConfigSchema, ...]:
|
||||
seen_names: set[str] = set()
|
||||
seen_model_keys: set[str] = set()
|
||||
schemas: List[ProviderConfigSchema] = []
|
||||
|
||||
for index, provider in enumerate(providers):
|
||||
if not isinstance(provider, Mapping):
|
||||
raise ValueError(f"Provider entry at index {index} must be an object")
|
||||
|
||||
normalized = normalize_runtime_provider_config(provider)
|
||||
name = normalized["name"]
|
||||
if name in seen_names:
|
||||
raise ValueError(f"Duplicate provider name detected: {name}")
|
||||
seen_names.add(name)
|
||||
schema = ProviderConfigSchema.from_mapping(provider)
|
||||
if schema.name in seen_names:
|
||||
raise ValueError(f"Duplicate provider name detected: {schema.name}")
|
||||
seen_names.add(schema.name)
|
||||
|
||||
for model in schema.model_list:
|
||||
model_key = f"{schema.name}/{model}"
|
||||
if model_key in seen_model_keys:
|
||||
raise ValueError(f"Duplicate provider model mapping detected: {model_key}")
|
||||
seen_model_keys.add(model_key)
|
||||
|
||||
schemas.append(schema)
|
||||
|
||||
return tuple(schemas)
|
||||
|
||||
|
||||
def validate_provider_definitions(providers: Sequence[Mapping[str, Any]]) -> None:
|
||||
build_provider_schemas(providers)
|
||||
@@ -1,72 +1,96 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Provider 管理器:负责加载配置、构建运行时对象并维护模型映射。"""
|
||||
"""Provider 管理器:负责加载配置、构建运行时对象并暴露只读注册表接口。"""
|
||||
|
||||
import hashlib
|
||||
import importlib.util
|
||||
import inspect
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional
|
||||
from typing import Dict, Optional, Sequence
|
||||
|
||||
from ..application.app_context import AppContext, Logger
|
||||
from ..external import LLMProvider
|
||||
from ..hooks import HookModule
|
||||
from .provider_config import normalize_runtime_provider_config
|
||||
from .provider_config import ProviderConfigSchema, ProviderRuntimeView, RuntimeProviderSpec
|
||||
|
||||
|
||||
class ProviderManager:
|
||||
"""管理模型到 provider 的映射关系。"""
|
||||
"""管理模型到 provider 的运行时映射关系。"""
|
||||
|
||||
def __init__(self, ctx: AppContext):
|
||||
self._base_dir = ctx.root_path.resolve()
|
||||
self._logger: Logger = ctx.logger
|
||||
self._provider_by_model: Dict[str, LLMProvider] = {}
|
||||
self._provider_names: set[str] = set()
|
||||
self._provider_by_name: Dict[str, LLMProvider] = {}
|
||||
self._provider_views_by_name: Dict[str, ProviderRuntimeView] = {}
|
||||
self._hook_cache: Dict[str, Optional[HookModule]] = {}
|
||||
|
||||
def load_providers(self, providers_config: List[Dict[str, Any]]) -> None:
|
||||
def load_providers(self, providers_config: Sequence[ProviderConfigSchema]) -> None:
|
||||
"""重载 provider 配置。"""
|
||||
self._provider_by_model.clear()
|
||||
self._provider_names.clear()
|
||||
self._provider_by_name.clear()
|
||||
self._provider_views_by_name.clear()
|
||||
self._hook_cache.clear()
|
||||
|
||||
for provider_cfg in providers_config:
|
||||
self._load_provider(provider_cfg)
|
||||
|
||||
def _load_provider(self, config: Dict[str, Any]) -> None:
|
||||
def get_provider_for_model(self, model_name: str) -> Optional[LLMProvider]:
|
||||
"""按模型 key 查询运行时 provider。"""
|
||||
return self._provider_by_model.get(model_name)
|
||||
|
||||
def list_model_names(self) -> tuple[str, ...]:
|
||||
"""返回当前已注册的模型 key 列表。"""
|
||||
return tuple(sorted(self._provider_by_model.keys()))
|
||||
|
||||
def get_provider_view(self, provider_name: str) -> Optional[ProviderRuntimeView]:
|
||||
"""按 provider 名称返回只读运行时视图。"""
|
||||
return self._provider_views_by_name.get(str(provider_name).strip())
|
||||
|
||||
def list_provider_views(self) -> tuple[ProviderRuntimeView, ...]:
|
||||
"""返回所有 provider 的只读运行时视图。"""
|
||||
return tuple(
|
||||
self._provider_views_by_name[name]
|
||||
for name in sorted(self._provider_views_by_name.keys())
|
||||
)
|
||||
|
||||
def has_model(self, model_name: str) -> bool:
|
||||
"""判断模型 key 是否已注册。"""
|
||||
return model_name in self._provider_by_model
|
||||
|
||||
def _load_provider(self, config: ProviderConfigSchema) -> None:
|
||||
"""加载单个 provider,并注册其模型映射。"""
|
||||
normalized = normalize_runtime_provider_config(config)
|
||||
name = normalized['name']
|
||||
model_list = normalized['model_list']
|
||||
spec = RuntimeProviderSpec.from_schema(config)
|
||||
if spec.name in self._provider_by_name:
|
||||
raise ValueError(f"Duplicate provider name detected: {spec.name}")
|
||||
|
||||
if name in self._provider_names:
|
||||
raise ValueError(f'Duplicate provider name detected: {name}')
|
||||
|
||||
if not model_list:
|
||||
self._logger.warning("Provider '%s' skipped: model_list is empty", name)
|
||||
if not spec.model_list:
|
||||
self._logger.warning("Provider '%s' skipped: model_list is empty", spec.name)
|
||||
return
|
||||
|
||||
provider = LLMProvider(
|
||||
name=name,
|
||||
api=normalized['api'],
|
||||
api_key=normalized['api_key'],
|
||||
model_list=model_list,
|
||||
proxy=normalized['proxy'],
|
||||
timeout_seconds=normalized['timeout_seconds'],
|
||||
max_retries=normalized['max_retries'],
|
||||
verify_ssl=normalized['verify_ssl'],
|
||||
hook=self._load_hook(normalized['hook']),
|
||||
name=spec.name,
|
||||
api=spec.api,
|
||||
api_key=spec.api_key,
|
||||
model_list=spec.model_list,
|
||||
proxy=spec.proxy,
|
||||
timeout_seconds=spec.timeout_seconds,
|
||||
max_retries=spec.max_retries,
|
||||
verify_ssl=spec.verify_ssl,
|
||||
hook=self._load_hook(spec.hook),
|
||||
)
|
||||
|
||||
for model in model_list:
|
||||
key = f'{name}/{model}'
|
||||
if key in self._provider_by_model:
|
||||
raise ValueError(f'Duplicate provider model mapping detected: {key}')
|
||||
self._provider_by_model[key] = provider
|
||||
self._provider_by_name[spec.name] = provider
|
||||
self._provider_views_by_name[spec.name] = ProviderRuntimeView.from_spec(spec)
|
||||
|
||||
self._provider_names.add(name)
|
||||
self._logger.info('Loaded provider: %s, models: %s', name, model_list)
|
||||
for model in spec.model_list:
|
||||
model_key = f"{spec.name}/{model}"
|
||||
if model_key in self._provider_by_model:
|
||||
raise ValueError(f"Duplicate provider model mapping detected: {model_key}")
|
||||
self._provider_by_model[model_key] = provider
|
||||
|
||||
self._logger.info("Loaded provider: %s, models: %s", spec.name, list(spec.model_list))
|
||||
|
||||
def _load_hook(self, hook_path: Optional[str]) -> Optional[HookModule]:
|
||||
"""按路径加载 hook 模块,并做缓存复用。"""
|
||||
@@ -76,14 +100,14 @@ class ProviderManager:
|
||||
if Path(hook_path).is_absolute():
|
||||
hook_file = Path(hook_path).resolve()
|
||||
else:
|
||||
hook_file = (self._base_dir / 'hooks' / hook_path).resolve()
|
||||
hook_file = (self._base_dir / "hooks" / hook_path).resolve()
|
||||
|
||||
cache_key = str(hook_file)
|
||||
if cache_key in self._hook_cache:
|
||||
return self._hook_cache[cache_key]
|
||||
|
||||
if not hook_file.exists():
|
||||
self._logger.warning('Hook file not found: %s', hook_file)
|
||||
self._logger.warning("Hook file not found: %s", hook_file)
|
||||
self._hook_cache[cache_key] = None
|
||||
return None
|
||||
|
||||
@@ -96,24 +120,24 @@ class ProviderManager:
|
||||
|
||||
hook_name = hook_file.stem
|
||||
normalized_path = hook_file.resolve().as_posix().lower()
|
||||
path_hash = hashlib.sha1(normalized_path.encode('utf-8')).hexdigest()[:12]
|
||||
module_name = f'hook_{hook_name}_{path_hash}'
|
||||
path_hash = hashlib.sha1(normalized_path.encode("utf-8")).hexdigest()[:12]
|
||||
module_name = f"hook_{hook_name}_{path_hash}"
|
||||
spec = importlib.util.spec_from_file_location(module_name, hook_file)
|
||||
if spec and spec.loader:
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
hook_class = getattr(module, 'Hook', None)
|
||||
hook_class = getattr(module, "Hook", None)
|
||||
if hook_class is None or not inspect.isclass(hook_class):
|
||||
self._logger.error('Hook file must export a class named Hook: %s', hook_file)
|
||||
self._logger.error("Hook file must export a class named Hook: %s", hook_file)
|
||||
self._hook_cache[cache_key] = None
|
||||
return None
|
||||
|
||||
hook_instance = hook_class()
|
||||
self._hook_cache[cache_key] = hook_instance
|
||||
self._logger.info('Hook loaded successfully: %s', hook_file)
|
||||
self._logger.info("Hook loaded successfully: %s", hook_file)
|
||||
return hook_instance
|
||||
except Exception as exc:
|
||||
self._logger.error('Failed to load hook %s: %s', hook_file, exc)
|
||||
self._logger.error("Failed to load hook %s: %s", hook_file, exc)
|
||||
finally:
|
||||
if path_inserted:
|
||||
try:
|
||||
@@ -123,9 +147,3 @@ class ProviderManager:
|
||||
|
||||
self._hook_cache[cache_key] = None
|
||||
return None
|
||||
|
||||
def find_provider_by_model(self, model_name: str) -> Optional[LLMProvider]:
|
||||
return self._provider_by_model.get(model_name)
|
||||
|
||||
def get_all_models(self) -> List[str]:
|
||||
return sorted(self._provider_by_model.keys())
|
||||
Vendored
+16
-23
@@ -2,42 +2,35 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""LLM provider 运行时对象。"""
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, Dict, Optional
|
||||
|
||||
from ..hooks import HookContext, HookModule
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class LLMProvider:
|
||||
"""封装 provider 配置与请求阶段 hook 调用。"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
name: str,
|
||||
api: str,
|
||||
api_key: Optional[str] = None,
|
||||
model_list: Optional[list] = None,
|
||||
proxy: Optional[str] = None,
|
||||
timeout_seconds: int = 300,
|
||||
max_retries: int = 3,
|
||||
verify_ssl: bool = False,
|
||||
hook: Optional[HookModule] = None,
|
||||
):
|
||||
self.name = name
|
||||
self.api = api
|
||||
self.api_key = api_key
|
||||
self.model_list = list(model_list or [])
|
||||
self.proxy = proxy
|
||||
self.timeout_seconds = timeout_seconds
|
||||
self.max_retries = max_retries
|
||||
self.verify_ssl = verify_ssl
|
||||
self.hook = hook
|
||||
name: str
|
||||
api: str
|
||||
api_key: Optional[str] = None
|
||||
model_list: tuple[str, ...] = ()
|
||||
proxy: Optional[str] = None
|
||||
timeout_seconds: int = 300
|
||||
max_retries: int = 3
|
||||
verify_ssl: bool = False
|
||||
hook: Optional[HookModule] = None
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
object.__setattr__(self, "model_list", tuple(self.model_list))
|
||||
|
||||
def apply_header_hook(self, ctx: HookContext, headers: Dict[str, str]) -> Dict[str, str]:
|
||||
if self.hook and hasattr(self.hook, 'header_hook'):
|
||||
if self.hook and hasattr(self.hook, "header_hook"):
|
||||
return self.hook.header_hook(ctx, headers)
|
||||
return headers
|
||||
|
||||
def apply_input_body_hook(self, ctx: HookContext, body: Dict[str, Any]) -> Dict[str, Any]:
|
||||
if self.hook and hasattr(self.hook, 'input_body_hook'):
|
||||
if self.hook and hasattr(self.hook, "input_body_hook"):
|
||||
return self.hook.input_body_hook(ctx, body)
|
||||
return body
|
||||
@@ -78,7 +78,7 @@ class ProxyController:
|
||||
request_data["stream_options"] = stream_options
|
||||
|
||||
model_name = request_data["model"]
|
||||
provider = self._provider_manager.find_provider_by_model(model_name)
|
||||
provider = self._provider_manager.get_provider_for_model(model_name)
|
||||
if not provider:
|
||||
self._logger.warning(f"Proxy rejected: unknown model={model_name!r}")
|
||||
return (
|
||||
@@ -138,7 +138,7 @@ class ProxyController:
|
||||
def list_models(self) -> Response:
|
||||
"""返回当前可用模型列表。"""
|
||||
try:
|
||||
data = [{"id": model_key} for model_key in self._provider_manager.get_all_models()]
|
||||
data = [{"id": model_key} for model_key in self._provider_manager.list_model_names()]
|
||||
return jsonify({"object": "list", "data": data})
|
||||
except Exception as exc:
|
||||
self._logger.error(f"Error listing models: {exc}")
|
||||
|
||||
@@ -698,8 +698,8 @@
|
||||
throw new Error(result.error || '保存 provider 失败');
|
||||
}
|
||||
editingProviderName = result.name;
|
||||
providerModalInstance?.hide();
|
||||
showMessage('Provider 保存成功,配置已重载', 'success');
|
||||
fillForm(result);
|
||||
await loadProviders();
|
||||
} catch (error) {
|
||||
console.error('保存 provider 失败:', error);
|
||||
|
||||
@@ -4,11 +4,11 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Callable, Dict, List, Optional
|
||||
from typing import Any, Callable, Dict, List, Optional
|
||||
|
||||
from ..application.app_context import AppContext
|
||||
from ..config.provider_config import (
|
||||
normalize_provider_payload,
|
||||
ProviderConfigSchema,
|
||||
validate_provider_definitions,
|
||||
)
|
||||
|
||||
@@ -33,10 +33,11 @@ class ProviderService:
|
||||
def create_provider(self, payload: Dict[str, Any]) -> Dict[str, Any]:
|
||||
config = self._config_manager.get_raw_config()
|
||||
providers = self._extract_providers(config)
|
||||
normalized = normalize_provider_payload(payload)
|
||||
provider_config = ProviderConfigSchema.from_payload(payload)
|
||||
normalized = provider_config.to_mapping()
|
||||
|
||||
if self._find_provider(providers, normalized['name']):
|
||||
raise ValueError(f"Provider already exists: {normalized['name']}")
|
||||
if self._find_provider(providers, provider_config.name):
|
||||
raise ValueError(f"Provider already exists: {provider_config.name}")
|
||||
|
||||
providers.append(normalized)
|
||||
self._save_providers(config, providers)
|
||||
@@ -45,15 +46,16 @@ class ProviderService:
|
||||
def update_provider(self, current_name: str, payload: Dict[str, Any]) -> Dict[str, Any]:
|
||||
config = self._config_manager.get_raw_config()
|
||||
providers = self._extract_providers(config)
|
||||
normalized = normalize_provider_payload(payload)
|
||||
provider_config = ProviderConfigSchema.from_payload(payload)
|
||||
normalized = provider_config.to_mapping()
|
||||
|
||||
target = self._find_provider(providers, current_name)
|
||||
if target is None:
|
||||
raise ValueError(f'Provider not found: {current_name}')
|
||||
|
||||
duplicate = self._find_provider(providers, normalized['name'])
|
||||
duplicate = self._find_provider(providers, provider_config.name)
|
||||
if duplicate is not None and duplicate is not target:
|
||||
raise ValueError(f"Provider already exists: {normalized['name']}")
|
||||
raise ValueError(f"Provider already exists: {provider_config.name}")
|
||||
|
||||
providers[providers.index(target)] = normalized
|
||||
self._save_providers(config, providers)
|
||||
|
||||
Reference in new issue
Block a user