feat: add tool-driven RAG agents

This commit is contained in:
kelriclink committed 2026-07-03 00:50:49 +08:00
1 parent 6524333c42
commit 40be75fc35
14 files changed
+1175 -175

No files matched your search

+52 -17
View File
@@ -2,16 +2,19 @@
模块化视频处理流水线:内容接入 -> 字幕/翻译 -> 压制/封装 -> 可选投稿到哔哩哔哩。
当前重点是**带 RAG Agent 的视频字幕翻译**:在翻译每个字幕 block 之前,系统会先判断哪些词真的需要知识库或联网查证,再把可信术语和背景知识注入翻译 prompt。目标不是把所有词都入库,而是解决 YouTube、动漫、游戏实况、技术讲解这类视频里“模型缺上下文”的问题。
详细规格见 [docs/PROJECT_SPEC.md](docs/PROJECT_SPEC.md)。
## 显著功能
- **RAG 翻译 Agent**:字幕翻译前先扫描当前 block 和前文 summary,判断哪些术语真的需要查知识库或联网检索,减少把基础词、局部变量写入长期知识库。
- **主 Agent + 子 Agent 翻译 RAG**:字幕翻译前由主 Agent 扫描当前 block 和前文 summary,决定哪些术语需要研究;每个术语由独立子 Agent 维护 observation/evidence 上下文并选择工具。
- **PostgreSQL + pgvector 知识库**:术语、译法、来源、置信度和使用次数统一入库;支持向量检索、手动新增/删除条目、重建 embedding。
- **Wikipedia Tool + SearXNG Tool**:Agent 优先查固定 English Wikipedia API,必要时再调用自建 SearXNG,并可读取网页正文作为证据。
- **并发术语研究 Agent**:可在 `Settings -> Translate` 设置同时研究的术语数量;每个 Agent 都有独立工具调用 trace。
- **真正的工具调用循环**:子 Agent 可按需调用 `rag_lookup`、`wiki_search`、`search_web`、`fetch_url`、`finish`,而不是固定按 wiki -> search 的流水线机械执行。
- **Wikipedia Tool + SearXNG Tool**:固定 English Wikipedia API 用于通用百科知识;自建 SearXNG 用于补充网络搜索;搜索摘要不足时可继续读取网页正文。
- **并发术语研究 Agent**:可在 `Settings -> Translate` 设置并发数和超时;主 Agent 等待子 Agent 返回结构化结果后再合并进当前 block 翻译。
- **保守入库策略**:summarize -> verify -> write_rag;只有 verifier 判断来源支撑、上下文一致且置信度足够时才正式入库,否则标记为 pending、context_only 或 skipped。
- **独立 embedding 配置**:翻译 API key/base URL 与 embedding API key/base URL 分开;embedding 支持 OpenAI 兼容服务和本地模型。
- **独立模型配置**:翻译大模型、embedding 模型的 API key/base URL 分开;翻译大模型支持单独设置失败重试次数;embedding 支持 OpenAI 兼容服务和本地模型。
- **可观测 Dashboard**:显示任务状态、CPU/内存/Intel GPU 资源监控,以及 RAG Agent 当前运行状态和历史对话流。
- **完整 Web 配置**:ASR、翻译、RAG、YouTube、Bilibili、自动模式、知识库均可在 Web UI 中配置,减少部署环境变量依赖。
@@ -26,7 +29,7 @@
- Bilibili 投稿支持在 Web UI 中保存 Cookie、测试登录、维护默认投稿模板。
- Web UI 支持浅色 / 暗色模式切换,Dashboard 增加 CPU、内存和 Intel GPU 资源监控。
- 字幕翻译支持真正的 RAG 术语知识库:可使用 PostgreSQL + pgvector 检索术语,支持本地 embedding 模型、手动下载模型、重建向量,以及 Agent 自动发现/搜索/验证/入库术语。
- RAG Agent 支持固定 Wikipedia Tool、SearXNG 搜索、网页正文读取、独立 verifier、保守入库策略,并可在 Dashboard 查看每个 Agent 的工具调用 trace。
- RAG Agent 已改为主 Agent / 子 Agent 结构:Dashboard 可查看主 Agent、其下属子 Agent、每一步工具调用、耗时、模型、错误类型和 JSON 输入输出。
## 架构概览
@@ -134,7 +137,27 @@ S3_USE_SSL=false
docker compose -f docker-compose.yml --env-file .env up --build -d
```
### 5. 首次登录
### 5. 使用已导出的镜像部署
如果你从 `dist/docker-images/videoroll-prod-app-web.tar` 部署:
```bash
docker load -i dist/docker-images/videoroll-prod-app-web.tar
docker compose -f docker-compose.yml --env-file .env up -d
```
该 tar 只包含业务镜像:
- `videoroll:prod`
- `videoroll-web:prod`
Redis、MinIO、PostgreSQL 仍按 compose 或你的现有部署提供。导出后可用 sha256 文件校验:
```bash
sha256sum -c dist/docker-images/videoroll-prod-app-web.tar.sha256
```
### 6. 首次登录
打开 Web UI:
@@ -328,17 +351,17 @@ RAG 用于解决视频字幕翻译中的上下文和术语问题,例如游戏
```text
字幕片段
+ 前文翻译 summary
-> RAG gate 判断是否需要检索
-> 术语发现与噪声过滤
-> 主 Agent 创建 rag_master run
-> RAG gate 判断哪些术语需要检索
-> 查询 PostgreSQL/pgvector 知识库
-> 未命中时并发启动术语研究 Agent
-> 优先调用 Wikipedia Tool
-> 必要时调用 SearXNG 搜索
-> 必要时读取网页正文
-> LLM 总结候选术语
-> 未命中时并发启动子 Agent
-> 子 Agent 自己决定调用 rag_lookup / wiki_search / search_web / fetch_url / finish
-> 子 Agent 收集证据并总结候选术语
-> verifier 检查来源和上下文
-> 写入知识库或标记为 context_only / skipped
-> 翻译时注入 rag_context
-> 子 Agent 返回 AgentResearchResult
-> 主 Agent 汇总 term_cards / knowledge_cards / hits
-> 翻译时注入 RagContext
```
关键行为:
@@ -347,8 +370,20 @@ RAG 用于解决视频字幕翻译中的上下文和术语问题,例如游戏
- 常见基础术语(如 `truth table`)可以作为本次翻译提示,但默认不自动污染长期知识库。
- 自动正式入库要求 verifier 通过、上下文一致、有有效外部来源,并达到较高置信度;否则进入待审核或跳过。
- Wikipedia Tool 使用固定 English Wikipedia API,不需要额外配置 wiki 地址;它适合通用百科类专有名词,不替代垂直领域资料库。
- 可在 `Settings -> Translate` 调整 Agent 并发数和超时时间。并发越高,术语补全越快,但外部 API、LLM 和搜索服务压力也越大。
- Dashboard 中可以点击 RAG Agent 查看每步工具调用、耗时、模型、错误类型和 JSON 输入输出。
- SearXNG Tool 使用 base URL 配置,系统自动拼接 `/search?q=...&format=json`,并过滤搜索引擎 About/Preferences 等内部页面。
- 可在 `Settings -> Translate` 调整 Agent 并发数、Agent 超时、大模型失败重试次数。并发越高,术语补全越快,但外部 API、LLM 和搜索服务压力也越大。
- Dashboard 中可以点击主 Agent 查看下属子 Agent;点击子 Agent 可查看它自己的工具调用对话流。
子 Agent 返回主 Agent 的数据是结构化结果,而不是把整段 trace 直接塞回 prompt:
```text
AgentResearchResult
- hit: 成功写入或命中的长期知识库条目
- context_card: 不适合长期入库但当前 block 可用的临时术语提示
- normalized_term: 去重用的规范化术语
```
主 Agent 会把这些结果合并成 `RagContext`,后续翻译 prompt 只注入精简后的术语卡片和知识卡片,避免长 trace 拖慢翻译。
PostgreSQL 需要启用 pgvector:
@@ -368,7 +403,7 @@ https://search.example.com
Embedding 支持两种方式:
- OpenAI 兼容 embedding:填写 provider 支持的 embedding 模型名。
- OpenAI 兼容 embedding:填写 provider 支持的 embedding 模型名、base URL 和独立 API key。
- 本地 embedding:在 `Settings -> Translate` 中选择本地 provider,可手动下载模型到 `data/models`,并选择 CPU / OpenVINO GPU 等设备。
更换 embedding 模型后,已有知识库向量不会静默重建;需要在 `Settings -> Translate` 中点击重建按钮,避免模型切换时旧向量和新向量混用。
+24 -12
View File
@@ -8,34 +8,45 @@
## 1. 当前显著能力
当前实现已经从“普通字幕翻译流水线”升级为带可观测 RAG Agent 的视频翻译系统,重点能力如下:
当前实现已经从“普通字幕翻译流水线”升级为带可观测 RAG Agent 的视频翻译系统。重点不是把所有词都查一遍,而是在翻译前判断“哪些词确实需要外部知识”,再把可信知识注入翻译 prompt。
- **字幕翻译 RAG gate**:每个翻译 block 开始前,LLM 先结合当前字幕和前文 summary 判断哪些术语真的需要检索,避免把基础词、局部变量、普通短语都塞进知识库。
- **PostgreSQL + pgvector 知识库**:保存术语、译法、领域、别名、解释、来源、置信度、状态和 embedding,翻译时按向量相似度召回并注入上下文。
- **单 Agent + 多 Tool 架构**:术语研究 Agent 统一调度 RAG、Wikipedia、SearXNG、网页读取、总结、验证和写库,不一开始拆成多个松散 Agent。
- **主 Agent + 子 Agent 架构**:每个字幕 block 创建一个 `rag_master` 主 Agent;主 Agent 负责术语 gate、调度并发子 Agent、汇总结果;每个术语研究子 Agent 维护自己的 observation/evidence 上下文。
- **工具调用循环**:子 Agent 可按需调用 `rag_lookup`、`wiki_search`、`search_web`、`fetch_url`、`finish`。工具选择由 LLM 根据当前 observation 决定,程序只做超时、重试、fallback 和写库安全控制。
- **Wikipedia Tool**:固定使用 English Wikipedia API,适合百科型专有名词和通用背景知识;不用在配置里维护各种不通用的 wiki 地址。
- **SearXNG Tool**:可接入自建 SearXNG,配置填写 base URL,系统自动请求 `/search?q=...&format=json` 并过滤搜索引擎自身页面等无效结果。
- **网页正文读取 Tool**:当搜索摘要不足时,Agent 可以打开候选链接抽取正文,再交给 LLM 总结证据。
- **Verifier 入库保护**:候选术语会经过 verifier 判断来源是否支撑、是否和字幕上下文一致、置信度是否足够;不满足条件时不会污染长期知识库。
- **并发术语研究**:翻译设置里可调整 RAG Agent 并发数和超时时间,让多个需要查证的术语同时研究,最后合并回当前翻译上下文。
- **并发术语研究**:翻译设置里可调整 RAG Agent 并发数和超时时间,让多个需要查证的术语同时研究;子 Agent 结束后返回结构化 `AgentResearchResult`,由主 Agent 合并回当前翻译上下文。
- **独立 embedding 配置**:翻译模型和 embedding 模型的 API key/base URL 分开;embedding 支持 OpenAI 兼容接口和本地模型目录。
- **Web 管理与可观测性**:知识库页面支持查看、新增、删除条目;Dashboard 可查看资源监控、正在运行的 Agent、历史 trace 和每步 JSON 输入输出。
- **独立大模型重试配置**:翻译大模型请求的失败重试次数可在翻译设置中单独配置,不和 embedding 请求混用。
- **Web 管理与可观测性**:知识库页面支持查看、新增、删除条目;Dashboard 可查看资源监控、主 Agent/子 Agent 运行树、历史 trace 和每步 JSON 输入输出。
核心翻译流程:
```text
字幕 block + 前文 summary
-> 创建 rag_master 主 Agent
-> RAG gate 判断需要检索的术语
-> 查询 pgvector 知识库
-> 未命中时并发启动术语研究 Agent
-> Wikipedia / SearXNG / fetch tools 收集证据
-> 先查询 pgvector 知识库
-> 未命中时并发启动术语研究子 Agent
-> 子 Agent 按 observation 决定 rag/wiki/search/fetch/finish
-> LLM summarize 生成候选译法
-> verifier 校验来源和上下文
-> 写入知识库或标记为 pending/context_only/skipped
-> 将命中知识和本次研究结果注入翻译 prompt
-> 子 Agent 返回 hit/context_card
-> 主 Agent 汇总 RagContext
-> 将精简后的术语卡片和知识卡片注入翻译 prompt
-> 输出当前 block 翻译并更新 summary
```
子 Agent 的 trace 用于调试和审计,不直接塞回翻译 prompt。最终进入翻译 prompt 的是主 Agent 汇总后的结构化上下文:
- `term_cards`:术语、译法、领域、说明、来源、置信度。
- `knowledge_cards`:文档型背景知识。
- `hits`:原始知识库命中,用于记录匹配和使用次数。
---
## 2. 总体拆分:三大模块 + 一个编排层(Orchestrator)
@@ -169,7 +180,8 @@
- 输入:`segments.json` 或 `subtitle_src.srt`
- 输出:`subtitle_zh.srt`
- 可插拔 provider:LLM / 翻译引擎。
- 翻译前可启用 RAG gate:结合当前 block 和前文 summary 发现需要查询的术语,先查 PostgreSQL/pgvector,未命中时由 Agent 调用 Wikipedia、SearXNG 和网页读取工具补充证据。
- 翻译前可启用 RAG gate:结合当前 block 和前文 summary 发现需要查询的术语;主 Agent 先查 PostgreSQL/pgvector,未命中时并发启动术语研究子 Agent。
- 子 Agent 通过工具调用循环自行决定下一步:本地 RAG、Wikipedia、SearXNG、网页正文读取或结束研究。
- RAG 结果不会无条件写入长期知识库;必须经过 verifier 校验来源、上下文一致性和置信度。
- 支持手动知识库条目、自动学习条目、context_only 临时提示和 pending 待确认条目。
@@ -347,7 +359,7 @@
### 8.1 Web 管控台核心功能
- 任务列表:状态筛选、搜索、批量重试/取消
- 任务详情:查看每 stage 的日志与产物;字幕文件在线预览/编辑;投稿 meta 编辑;审核通过/驳回
- Dashboard:任务状态、队列概览、CPU/内存/Intel GPU 资源监控、RAG Agent 运行状态和历史 trace
- Dashboard:任务状态、队列概览、CPU/内存/Intel GPU 资源监控、RAG 主 Agent/子 Agent 运行树和历史 trace
- 知识库管理:查看、新增、删除 RAG 条目;触发 embedding 重建
- 模板中心:标题/简介/标签/分区/动态文案模板,支持变量(如 `{source_title}`、`{publish_date}`)
- 账号与凭据:B 站凭据录入与轮换提醒;权限控制
@@ -383,8 +395,8 @@
**M5:RAG Agent 翻译增强**
- PostgreSQL + pgvector 知识库
- RAG gate + 术语噪声过滤
- Wikipedia / SearXNG / fetch tools
- 主 Agent gate + 术语噪声过滤
- 子 Agent 工具循环:RAG / Wikipedia / SearXNG / fetch URL
- verifier 保守入库
- Dashboard trace 与知识库管理页面
+8 -3
View File
@@ -21,6 +21,8 @@ class OpenAIChatConfig:
model: str
temperature: float = 0.2
timeout_seconds: float = 60.0
max_retries: int = 3
embedding_dimensions: int | None = None
def openai_chat_config_from_settings(settings: Mapping[str, Any]) -> OpenAIChatConfig:
@@ -30,6 +32,7 @@ def openai_chat_config_from_settings(settings: Mapping[str, Any]) -> OpenAIChatC
model=str(settings.get("openai_model") or "").strip(),
temperature=float(settings.get("openai_temperature") or 0.2),
timeout_seconds=float(settings.get("openai_timeout_seconds") or 60.0),
max_retries=max(1, min(10, int(settings.get("openai_max_retries") or 3))),
)
@@ -180,7 +183,7 @@ def request_openai_json_object(
user_prompt: str,
format_retry_notice: str = "注意:上一次输出不符合 JSON/结构要求,请严格按 JSON 输出。",
format_retries: int = 2,
network_retries: int = 3,
network_retries: int | None = None,
client: httpx.Client | None = None,
) -> dict[str, Any]:
if client is not None:
@@ -191,7 +194,7 @@ def request_openai_json_object(
user_prompt=user_prompt,
format_retry_notice=format_retry_notice,
format_retries=format_retries,
network_retries=network_retries,
network_retries=max(1, int(network_retries if network_retries is not None else config.max_retries)),
)
with create_openai_http_client(config.timeout_seconds) as owned_client:
@@ -202,7 +205,7 @@ def request_openai_json_object(
user_prompt=user_prompt,
format_retry_notice=format_retry_notice,
format_retries=format_retries,
network_retries=network_retries,
network_retries=max(1, int(network_retries if network_retries is not None else config.max_retries)),
)
@@ -223,6 +226,8 @@ def request_openai_embedding(
url = build_openai_embeddings_url(config.base_url)
headers = {"Authorization": f"Bearer {config.api_key}"}
req = {"model": config.model, "input": source}
if config.embedding_dimensions is not None and config.embedding_dimensions > 0:
req["dimensions"] = int(config.embedding_dimensions)
attempts_network = max(1, int(network_retries))
last_err: Exception | None = None
@@ -270,5 +270,6 @@ def embedding_settings_from_translate_settings(settings: dict[str, Any]) -> Embe
model=str(settings.get("rag_embedding_model") or "text-embedding-3-small").strip() or "text-embedding-3-small",
temperature=0.0,
timeout_seconds=embedding_timeout,
embedding_dimensions=max(1, min(4096, int(settings.get("rag_embedding_dimensions") or 1536))),
),
)
@@ -336,6 +336,7 @@ def get_translate_settings_view(
openai_model=cfg["openai_model"],
openai_temperature=cfg["openai_temperature"],
openai_timeout_seconds=cfg["openai_timeout_seconds"],
openai_max_retries=cfg["openai_max_retries"],
rag_enabled=cfg["rag_enabled"],
rag_top_k=cfg["rag_top_k"],
rag_min_score=cfg["rag_min_score"],
@@ -377,6 +378,7 @@ def put_translate_settings_view(
openai_model=cfg["openai_model"],
openai_temperature=cfg["openai_temperature"],
openai_timeout_seconds=cfg["openai_timeout_seconds"],
openai_max_retries=cfg["openai_max_retries"],
rag_enabled=cfg["rag_enabled"],
rag_top_k=cfg["rag_top_k"],
rag_min_score=cfg["rag_min_score"],
File diff suppressed because it is too large. Load diff
@@ -201,6 +201,7 @@ class TranslateSettingsRead(BaseModel):
openai_model: str
openai_temperature: float
openai_timeout_seconds: float
openai_max_retries: int = 3
rag_enabled: bool = False
rag_top_k: int = 8
@@ -237,6 +238,7 @@ class TranslateSettingsUpdate(BaseModel):
openai_model: Optional[str] = None
openai_temperature: Optional[float] = None
openai_timeout_seconds: Optional[float] = None
openai_max_retries: Optional[int] = Field(default=None, ge=1, le=10)
rag_enabled: Optional[bool] = None
rag_top_k: Optional[int] = Field(default=None, ge=0, le=30)
@@ -341,6 +343,7 @@ class AgentRunRead(BaseModel):
result: dict[str, Any] = Field(default_factory=dict)
error: str = ""
knowledge_item_id: Optional[uuid.UUID] = None
parent_agent_run_id: Optional[uuid.UUID] = None
started_at: datetime
finished_at: Optional[datetime] = None
created_at: datetime
@@ -65,6 +65,7 @@ def get_translate_settings(db: Session, defaults: SubtitleServiceSettings) -> di
"openai_model": str(openai.get("model") or defaults.openai_model),
"openai_temperature": float(openai.get("temperature") or defaults.openai_temperature),
"openai_timeout_seconds": float(openai.get("timeout_seconds") or defaults.openai_timeout_seconds),
"openai_max_retries": int(openai.get("max_retries") or stored.get("openai_max_retries") or 3),
"rag_enabled": bool(stored.get("rag_enabled") if "rag_enabled" in stored else defaults.rag_enabled),
"rag_top_k": int(stored.get("rag_top_k") or defaults.rag_top_k),
"rag_min_score": float(stored.get("rag_min_score") or defaults.rag_min_score),
@@ -148,6 +149,7 @@ def update_translate_settings(db: Session, defaults: SubtitleServiceSettings, up
("openai_model", "model"),
("openai_temperature", "temperature"),
("openai_timeout_seconds", "timeout_seconds"),
("openai_max_retries", "max_retries"),
]:
if key not in update:
continue
@@ -239,6 +241,11 @@ def update_translate_settings(db: Session, defaults: SubtitleServiceSettings, up
openai["timeout_seconds"] = float(openai.get("timeout_seconds"))
except Exception:
openai["timeout_seconds"] = defaults.openai_timeout_seconds
try:
if "max_retries" in openai and openai["max_retries"] is not None:
openai["max_retries"] = max(1, min(10, int(openai.get("max_retries") or 3)))
except Exception:
openai["max_retries"] = 3
row.value_json = stored
db.add(row)
+17
View File
@@ -201,6 +201,7 @@ def _ensure_pgvector_rag_tables(engine: Engine) -> None:
result JSONB NOT NULL DEFAULT '{}'::jsonb,
error TEXT NOT NULL DEFAULT '',
knowledge_item_id UUID REFERENCES translation_knowledge_items(id) ON DELETE SET NULL,
parent_agent_run_id UUID,
started_at TIMESTAMPTZ NOT NULL DEFAULT now(),
finished_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
@@ -209,6 +210,14 @@ def _ensure_pgvector_rag_tables(engine: Engine) -> None:
"""
)
)
conn.execute(
text(
"""
ALTER TABLE translation_agent_runs
ADD COLUMN IF NOT EXISTS parent_agent_run_id UUID
"""
)
)
conn.execute(
text(
"""
@@ -226,6 +235,14 @@ def _ensure_pgvector_rag_tables(engine: Engine) -> None:
"""
)
)
conn.execute(
text(
"""
CREATE INDEX IF NOT EXISTS ix_translation_agent_runs_parent
ON translation_agent_runs (parent_agent_run_id, updated_at DESC)
"""
)
)
conn.execute(
text(
"""
+7 -3
View File
@@ -61,10 +61,14 @@ export function Section({ children, className = "" }: PropsWithChildren<{ classN
return <div className={`vr-section ${className}`}>{children}</div>;
}
export function DataTable({ children }: PropsWithChildren) {
export function DataTable({
children,
className = "",
wrapClassName = "",
}: PropsWithChildren<{ className?: string; wrapClassName?: string }>) {
return (
<div className="vr-table-wrap">
<table className="vr-table">{children}</table>
<div className={`vr-table-wrap ${wrapClassName}`}>
<table className={`vr-table ${className}`}>{children}</table>
</div>
);
}
+68 -7
View File
@@ -56,6 +56,7 @@ type AgentRun = {
result: Record<string, unknown>;
error: string;
knowledge_item_id?: string | null;
parent_agent_run_id?: string | null;
started_at: string;
finished_at?: string | null;
created_at: string;
@@ -177,7 +178,17 @@ function AgentStep({ step, index }: { step: Record<string, unknown>; index: numb
);
}
function AgentRunDetail({ run, onClose }: { run: AgentRun; onClose: () => void }) {
function AgentRunDetail({
run,
childrenRuns = [],
onSelectRun,
onClose,
}: {
run: AgentRun;
childrenRuns?: AgentRun[];
onSelectRun: (run: AgentRun) => void;
onClose: () => void;
}) {
const translation = textValue(run.result?.translation);
const confidence = numberValue(run.result?.confidence);
const knowledgeStatus = textValue(run.result?.knowledge_status);
@@ -234,6 +245,27 @@ function AgentRunDetail({ run, onClose }: { run: AgentRun; onClose: () => void }
<pre className="max-h-64 max-w-full overflow-auto whitespace-pre rounded bg-slate-950 p-3 text-xs leading-relaxed text-slate-100">{prettyJson(run.result)}</pre>
</div>
) : null}
{childrenRuns.length ? (
<div className="min-w-0 rounded-md border border-slate-200 p-3">
<div className="mb-2 text-xs font-semibold text-slate-600">子 Agent</div>
<div className="max-h-64 space-y-1 overflow-auto pr-1">
{childrenRuns.map((child) => (
<button
key={child.id}
type="button"
className="block w-full rounded border border-slate-200 px-2 py-1.5 text-left hover:bg-slate-50"
onClick={() => onSelectRun(child)}
>
<div className="flex items-center justify-between gap-2">
<span className="min-w-0 truncate text-xs font-medium text-slate-800">{child.term || child.query || child.id}</span>
<span className={`shrink-0 rounded border px-1.5 py-0.5 text-[11px] ${agentStatusClass(child.status)}`}>{child.status}</span>
</div>
<div className="mt-1 truncate font-mono text-[11px] text-slate-500">{child.agent_type} · {formatDate(child.updated_at)}</div>
</button>
))}
</div>
</div>
) : null}
</div>
<div className="min-w-0">
<div className="mb-2 text-sm font-semibold text-slate-900">对话流</div>
@@ -248,7 +280,7 @@ function AgentRunDetail({ run, onClose }: { run: AgentRun; onClose: () => void }
);
}
function AgentRunCard({ run, onSelect }: { run: AgentRun; onSelect: (run: AgentRun) => void }) {
function AgentRunCard({ run, onSelect, childrenRuns = [] }: { run: AgentRun; onSelect: (run: AgentRun) => void; childrenRuns?: AgentRun[] }) {
const translation = textValue(run.result?.translation);
const confidence = numberValue(run.result?.confidence);
const knowledgeStatus = textValue(run.result?.knowledge_status);
@@ -268,6 +300,7 @@ function AgentRunCard({ run, onSelect }: { run: AgentRun; onSelect: (run: AgentR
{run.domain ? <div className="truncate">domain: {run.domain}</div> : null}
{run.query ? <div className="truncate">query: {run.query}</div> : null}
{opened ? <div>opened/read pages: {opened}</div> : null}
{childrenRuns.length ? <div>subagents: {childrenRuns.length}</div> : null}
{latestStep?.action ? <div className="truncate">latest: {String(latestStep.action)}</div> : null}
{translation ? (
<div className="truncate">
@@ -279,6 +312,16 @@ function AgentRunCard({ run, onSelect }: { run: AgentRun; onSelect: (run: AgentR
{failureCategory ? <div className="truncate text-amber-700">failure: {failureCategory}</div> : null}
{run.error ? <div className="line-clamp-2 text-rose-700">{run.error}</div> : null}
</div>
{childrenRuns.length ? (
<div className="mt-3 space-y-1 border-l border-slate-200 pl-3">
{childrenRuns.slice(0, 5).map((child) => (
<div key={child.id} className="flex items-center justify-between gap-2 text-xs text-slate-600">
<span className="min-w-0 truncate">{child.term || child.query || child.id}</span>
<span className={`shrink-0 rounded border px-1.5 py-0.5 ${agentStatusClass(child.status)}`}>{child.status}</span>
</div>
))}
</div>
) : null}
</button>
);
}
@@ -385,8 +428,19 @@ export default function DashboardPage() {
const total = tasks?.length ?? 0;
const failedTasks = useMemo(() => (tasks ?? []).filter((t) => t.status === "FAILED").slice(0, 6), [tasks]);
const runningTasks = useMemo(() => (tasks ?? []).filter((t) => runningStatuses.has(t.status)).slice(0, 6), [tasks]);
const runningAgents = useMemo(() => (agentRuns ?? []).filter((run) => run.status === "running"), [agentRuns]);
const recentFinishedAgents = useMemo(() => (agentRuns ?? []).filter((run) => run.status !== "running").slice(0, 6), [agentRuns]);
const agentChildren = useMemo(() => {
const out = new Map<string, AgentRun[]>();
for (const run of agentRuns ?? []) {
if (!run.parent_agent_run_id) continue;
const rows = out.get(run.parent_agent_run_id) ?? [];
rows.push(run);
out.set(run.parent_agent_run_id, rows);
}
return out;
}, [agentRuns]);
const topLevelAgents = useMemo(() => (agentRuns ?? []).filter((run) => !run.parent_agent_run_id), [agentRuns]);
const runningAgents = useMemo(() => topLevelAgents.filter((run) => run.status === "running"), [topLevelAgents]);
const recentFinishedAgents = useMemo(() => topLevelAgents.filter((run) => run.status !== "running").slice(0, 6), [topLevelAgents]);
const selectedAgent = useMemo(() => (agentRuns ?? []).find((run) => run.id === selectedAgentId) ?? null, [agentRuns, selectedAgentId]);
const publishedCount = counts.find(([status]) => status === "PUBLISHED")?.[1] ?? 0;
const failedCount = counts.find(([status]) => status === "FAILED")?.[1] ?? 0;
@@ -522,14 +576,14 @@ export default function DashboardPage() {
<div className="mb-2 text-xs font-semibold text-slate-600">正在工作</div>
<div className="space-y-2">
{runningAgents.length === 0 ? <div className="text-sm text-slate-500">暂无运行中的 agent。</div> : null}
{runningAgents.map((run) => <AgentRunCard key={run.id} run={run} onSelect={(item) => setSelectedAgentId(item.id)} />)}
{runningAgents.map((run) => <AgentRunCard key={run.id} run={run} childrenRuns={agentChildren.get(run.id) ?? []} onSelect={(item) => setSelectedAgentId(item.id)} />)}
</div>
</div>
<div>
<div className="mb-2 text-xs font-semibold text-slate-600">最近结果</div>
<div className="space-y-2">
{recentFinishedAgents.length === 0 ? <div className="text-sm text-slate-500">暂无 agent 结果。</div> : null}
{recentFinishedAgents.map((run) => <AgentRunCard key={run.id} run={run} onSelect={(item) => setSelectedAgentId(item.id)} />)}
{recentFinishedAgents.map((run) => <AgentRunCard key={run.id} run={run} childrenRuns={agentChildren.get(run.id) ?? []} onSelect={(item) => setSelectedAgentId(item.id)} />)}
</div>
</div>
</div>
@@ -648,7 +702,14 @@ export default function DashboardPage() {
</div>
) : null}
</div>
{selectedAgent ? <AgentRunDetail run={selectedAgent} onClose={() => setSelectedAgentId(null)} /> : null}
{selectedAgent ? (
<AgentRunDetail
run={selectedAgent}
childrenRuns={agentChildren.get(selectedAgent.id) ?? []}
onSelectRun={(run) => setSelectedAgentId(run.id)}
onClose={() => setSelectedAgentId(null)}
/>
) : null}
</div>
);
}
+1 -1
View File
@@ -235,7 +235,7 @@ export default function KnowledgeBasePage() {
{items.length === 0 ? (
<EmptyState>暂无知识条目</EmptyState>
) : (
<DataTable>
<DataTable wrapClassName="max-h-[34rem]">
<thead>
<tr>
<th className="py-2 pr-3 text-left">Type</th>
@@ -17,6 +17,7 @@ type TranslateSettings = {
openai_model: string;
openai_temperature: number;
openai_timeout_seconds: number;
openai_max_retries: number;
rag_enabled: boolean;
rag_top_k: number;
rag_min_score: number;
@@ -67,6 +68,7 @@ export default function SettingsTranslatePage() {
const [openaiModel, setOpenaiModel] = useState("gpt-4o-mini");
const [openaiTemperature, setOpenaiTemperature] = useState(0.2);
const [openaiTimeoutSeconds, setOpenaiTimeoutSeconds] = useState(60);
const [openaiMaxRetries, setOpenaiMaxRetries] = useState(3);
const [openaiApiKey, setOpenaiApiKey] = useState("");
const [ragEnabled, setRagEnabled] = useState(false);
@@ -119,6 +121,7 @@ export default function SettingsTranslatePage() {
setOpenaiModel(s.openai_model);
setOpenaiTemperature(s.openai_temperature);
setOpenaiTimeoutSeconds(s.openai_timeout_seconds);
setOpenaiMaxRetries(s.openai_max_retries ?? 3);
setRagEnabled(s.rag_enabled);
setRagTopK(s.rag_top_k);
setRagMinScore(s.rag_min_score);
@@ -161,6 +164,7 @@ export default function SettingsTranslatePage() {
openai_model: openaiModel,
openai_temperature: openaiTemperature,
openai_timeout_seconds: openaiTimeoutSeconds,
openai_max_retries: openaiMaxRetries,
rag_enabled: ragEnabled,
rag_top_k: ragTopK,
rag_min_score: ragMinScore,
@@ -397,6 +401,11 @@ export default function SettingsTranslatePage() {
<div className="mb-1 text-xs text-slate-600">openai_timeout_seconds</div>
<input type="number" min={1} className="w-full rounded border px-3 py-2 text-sm" value={openaiTimeoutSeconds} onChange={(e) => setOpenaiTimeoutSeconds(parseFloat(e.target.value || "1"))} />
</label>
<label className="block">
<div className="mb-1 text-xs text-slate-600">openai_max_retries</div>
<input type="number" min={1} max={10} className="w-full rounded border px-3 py-2 text-sm" value={openaiMaxRetries} onChange={(e) => setOpenaiMaxRetries(parseInt(e.target.value || "3", 10))} />
<div className="mt-1 text-xs text-slate-500">LLM 请求的网络/5xx 重试次数;不影响 embedding 请求。</div>
</label>
</div>
</Section>
+210 -4
View File
@@ -1,11 +1,17 @@
from __future__ import annotations
import json
import httpx
from videoroll.ai.client import OpenAIChatConfig
from videoroll.ai.client import request_openai_embedding
from videoroll.apps.subtitle_service.rag import (
_run_research_agents,
_fallback_search_queries,
_search_url_with_params,
build_knowledge_embedding_text,
build_rag_context,
embedding_model_key,
fetch_search_evidence,
fetch_wikipedia_evidence,
@@ -17,6 +23,7 @@ from videoroll.apps.subtitle_service.rag import (
search_knowledge,
should_research_term,
)
from videoroll.apps.subtitle_service.processing import Segment
from videoroll.apps.subtitle_service.embeddings import (
assert_embedding_dimensions,
embed_text,
@@ -220,7 +227,7 @@ def test_run_research_agents_uses_session_factory_for_parallel_terms(monkeypatch
from videoroll.apps.subtitle_service import rag as rag_module
created_sessions: list[object] = []
calls: list[tuple[object, str]] = []
calls: list[tuple[object, str, str | None]] = []
class _Session:
def close(self) -> None:
@@ -231,9 +238,9 @@ def test_run_research_agents_uses_session_factory_for_parallel_terms(monkeypatch
created_sessions.append(session)
return session
def fake_research(db, *, item, **_kwargs):
def fake_research(db, *, item, **kwargs):
term = item["term"]
calls.append((db, term))
calls.append((db, term, kwargs.get("parent_agent_run_id")))
return rag_module.AgentResearchResult(term=term, normalized_term=normalize_term(term))
monkeypatch.setattr(rag_module, "_research_discovered_term", fake_research)
@@ -259,13 +266,15 @@ def test_run_research_agents_uses_session_factory_for_parallel_terms(monkeypatch
previous_summary="",
existing_term_cards=[],
gate_duration_ms=1,
parent_agent_run_id="master-run",
task_id=None,
subtitle_job_id=None,
)
assert {item.term for item in results} == {"AWP", "Rush B"}
assert len(created_sessions) == 2
assert {id(db) for db, _term in calls} == {id(session) for session in created_sessions}
assert {id(db) for db, _term, _parent in calls} == {id(session) for session in created_sessions}
assert {parent for _db, _term, parent in calls} == {"master-run"}
def test_fallback_search_queries_include_definition_and_target_language() -> None:
@@ -371,6 +380,33 @@ def test_assert_embedding_dimensions() -> None:
raise AssertionError("expected dimension mismatch")
def test_openai_embedding_request_sends_dimensions() -> None:
requests: list[dict[str, object]] = []
def handler(request: httpx.Request) -> httpx.Response:
data = dict(json.loads(request.content.decode("utf-8")))
requests.append(data)
assert data["model"] == "Qwen3-Embedding-8B"
assert data["input"] == "纳米"
assert data["dimensions"] == 4096
return httpx.Response(200, json={"data": [{"embedding": [0.1, 0.2]}]})
client = httpx.Client(transport=httpx.MockTransport(handler))
vector = request_openai_embedding(
config=OpenAIChatConfig(
api_key="embed-key",
base_url="https://embedding.example/v1",
model="Qwen3-Embedding-8B",
embedding_dimensions=4096,
),
text="纳米",
client=client,
)
assert vector == [0.1, 0.2]
assert requests
class _Result:
def __init__(self, rows: list[object]) -> None:
self._rows = rows
@@ -408,6 +444,176 @@ class _FakeKnowledgeDb:
return _NestedTransaction()
def test_research_falls_back_to_search_when_wiki_verification_rejects(monkeypatch) -> None:
from videoroll.apps.subtitle_service import rag as rag_module
tool_calls: list[str] = []
steps: list[dict[str, object]] = []
finished: dict[str, object] = {}
monkeypatch.setattr(rag_module, "_start_agent_run", lambda *_args, **_kwargs: "run-1")
monkeypatch.setattr(rag_module, "_append_llm_step", lambda *_args, **_kwargs: None)
monkeypatch.setattr(rag_module, "_append_agent_step", lambda _db, _run_id, step: steps.append(step))
monkeypatch.setattr(
rag_module,
"_finish_agent_run",
lambda _db, _run_id, **kwargs: finished.update(kwargs),
)
monkeypatch.setattr(rag_module, "existing_term_norms", lambda *_args, **_kwargs: set())
monkeypatch.setattr(rag_module, "generate_search_queries_openai", lambda **_kwargs: ["hobby knife definition"])
decisions = iter(
[
{"action": "wiki_search", "query": "hobby knife definition", "reason": "start with encyclopedic source"},
{"action": "finish", "reason": "wiki evidence collected", "final_answer_ready": True},
]
)
monkeypatch.setattr(rag_module, "research_agent_next_action_openai", lambda **_kwargs: next(decisions))
def fake_wiki(*_args, **_kwargs):
tool_calls.append("wiki")
return [
{
"title": "Hobby knife",
"url": "https://en.wikipedia.org/wiki/Utility_knife",
"snippet": "A precision knife used for hobbies.",
"content": "A precision knife used for hobbies and model making.",
}
]
def fake_search(*_args, **_kwargs):
tool_calls.append("search")
return [
{
"title": "Hobby knife definition",
"url": "https://example.com/hobby-knife",
"snippet": "Hobby knives are precision cutting tools.",
"content": "Hobby knives are precision cutting tools used by model makers.",
}
]
monkeypatch.setattr(rag_module, "fetch_wikipedia_evidence", fake_wiki)
monkeypatch.setattr(rag_module, "fetch_search_evidence", fake_search)
monkeypatch.setattr(
rag_module,
"explain_term_from_evidence_openai",
lambda **_kwargs: {
"term": "hobby knife",
"translation": "模型刀",
"domain": "手工模型",
"aliases": [],
"description": "精细切割工具。",
"sources": [],
"confidence": 0.8,
},
)
monkeypatch.setattr(
rag_module,
"verify_glossary_entry_openai",
lambda **_kwargs: {
"supported": False,
"context_consistent": False,
"should_write": False,
"should_auto_approve": False,
"confidence": 0.0,
"reason": "unsupported",
"failure_category": "unsupported_translation_or_no_context",
},
)
result = rag_module._research_discovered_term(
object(), # type: ignore[arg-type]
item={"term": "hobby knife", "category": "domain_jargon", "need_rag": True, "need_search": True, "scope": "global"},
target_lang="zh",
rag_settings=rag_settings_from_translate_settings(
{
"rag_enabled": True,
"rag_auto_learn_terms": True,
"rag_wiki_enabled": True,
"rag_search_enabled": True,
"rag_search_url": "https://search.example/search",
"rag_embedding_dimensions": 2,
}
),
embedding_settings=embedding_settings_from_translate_settings({"rag_embedding_provider": "local", "rag_embedding_dimensions": 2}),
chat_config=OpenAIChatConfig(api_key="x", base_url="https://example.invalid/v1", model="demo"),
text_value="Use a hobby knife to cut the tape.",
llm_context="Use a hobby knife to cut the tape.",
previous_summary="",
existing_term_cards=[],
gate_duration_ms=1,
task_id=None,
subtitle_job_id=None,
)
assert result is not None
assert tool_calls == ["wiki", "search"]
assert any(step.get("action") == "evidence_tool_fallback" and step.get("tool") == "search" for step in steps)
assert finished["status"] == "skipped"
assert finished["result"]["tools_used"] == ["wikipedia", "search"] # type: ignore[index]
def test_build_rag_context_skips_research_for_existing_pending_term(monkeypatch) -> None:
from videoroll.apps.subtitle_service import rag as rag_module
research_items: list[dict[str, object]] = []
class _Db:
def execute(self, stmt: object, params: dict[str, object] | None = None) -> _Result:
sql = str(stmt)
if "SELECT DISTINCT normalized_term" in sql:
return _Result([_Row(normalized_term="hobby knife")])
return _Result([])
def rollback(self) -> None:
return None
monkeypatch.setattr(
rag_module,
"pretranslation_rag_gate_openai",
lambda *_args, **_kwargs: [
{
"term": "hobby knife",
"category": "domain_jargon",
"domain": "手工模型",
"need_rag": True,
"need_search": True,
"scope": "global",
"priority": 0.8,
"reason": "tool name",
}
],
)
monkeypatch.setattr(rag_module, "exact_term_hits", lambda *_args, **_kwargs: [])
monkeypatch.setattr(rag_module, "embed_text", lambda *_args, **_kwargs: [0.1, 0.2])
monkeypatch.setattr(rag_module, "assert_embedding_dimensions", lambda *_args, **_kwargs: None)
monkeypatch.setattr(rag_module, "search_knowledge", lambda *_args, **_kwargs: [])
def fake_run_research_agents(*_args, **kwargs):
research_items.extend(kwargs["items"])
return []
monkeypatch.setattr(rag_module, "_run_research_agents", fake_run_research_agents)
ctx = build_rag_context(
_Db(), # type: ignore[arg-type]
segments=[Segment(0, 1, "Use a hobby knife to cut the tape.")],
target_lang="zh",
rag_settings=rag_settings_from_translate_settings(
{
"rag_enabled": True,
"rag_auto_discover_terms": True,
"rag_auto_learn_terms": True,
"rag_embedding_dimensions": 2,
}
),
embedding_settings=embedding_settings_from_translate_settings({"rag_embedding_provider": "local", "rag_embedding_dimensions": 2}),
chat_config=OpenAIChatConfig(api_key="x", base_url="https://example.invalid/v1", model="demo"),
)
assert ctx.term_cards == []
assert research_items == []
def test_search_knowledge_filters_by_embedding_model() -> None:
db = _FakeKnowledgeDb([])
search_knowledge(