feat(tests): enhance AI service tests for knowledge independence and embedding model handling
- Added tests for AI service to ensure it operates independently of knowledge configuration, including scenarios with missing or corrupt embedding models. - Updated existing tests to prepare embedding models before execution to ensure consistent test environments. - Introduced new utility functions for managing embedding models in test setups. - Improved error handling in knowledge retrieval tests to verify behavior when embedding models are unavailable. - Refactored test harness to streamline embedding model preparation and ensure proper initialization before running tests.
This commit is contained in:
1 parent
5dd6cff103
commit
3bb3875eb1
38 files changed
+737
-385
No files matched your search
+1
-1
@@ -29,7 +29,7 @@ NCurator-Desktop 是本地优先的 Electron 桌面应用。文档解析、嵌
|
||||
|
||||
**检索**
|
||||
|
||||
向量搜索(语义)+ LanceDB FTS(关键词)+ RRF 融合。设置页「更新索引」进行中时,知识库工具直接失败;检查更新不拦截检索。
|
||||
向量搜索(语义)+ LanceDB FTS(关键词)+ RRF 融合。
|
||||
|
||||
### 分层与主链路
|
||||
|
||||
|
||||
+1
-1
@@ -30,7 +30,7 @@
|
||||
- 检查更新:读取当前真实数据,判断哪些文件或网页需要重新建立索引。
|
||||
- 更新索引:只处理可索引或失败资料,把内容写入后续检索可用的本地索引。
|
||||
- 网页资料:全局一份,由 URL 导入并抓取;`UNIQUE(normalized_url)`。
|
||||
- 向量模型:本地检索与知识库能力依赖的嵌入模型。
|
||||
- 向量模型:知识库工具与索引依赖的本地嵌入模型;工作空间和向量模型配置均不影响普通 AI run 启动。模型只由用户主动下载或恢复已有下载任务,KB / 索引不隐式下载。删除模型释放已加载运行时;后续 KB 调用返回错误,不终止聊天。
|
||||
|
||||
## 5. 需求规则
|
||||
|
||||
|
||||
@@ -53,7 +53,7 @@ PPT 普通预览和审核需完整显示中文及全角标点,换行测量与
|
||||
- Composer 附件不限制文件扩展名、文件大小、数量或目录大小;点击 `+` 直接打开可选择文件或目录的系统选择器,聊天区拖放接受文件和目录。文件及目录附件都保留源路径,不复制、不预解析目录,只把附件顺序、名称和绝对路径交给 AI 后续自行处理。附件卡片使用紧凑布局,仅图片显示缩略图;文件夹使用文件夹图标,音频和视频使用对应媒体图标。图片点击保持图片灯箱,支持右侧文件预览的文档点击打开右侧资源 Tab,文件夹继续用系统文件管理器打开,音视频及其他不支持右侧预览的文件定位到系统文件所在位置。粘贴文件因不保证有稳定源路径,保存副本到当前伙伴运行时目录。AI 工具侧仍有各自格式边界(如 `doc` 只读、`pptx` 可交给 PPT 工具读取和编辑、Excel 工具只写 `xlsx`),无法处理的文件会在 AI 工具层报错。
|
||||
- 其他格式不保证直接写回和回退。AI 可以在审批边界内尝试处理,但不属于受支持文件能力范围。
|
||||
- 审核权限下,终端副作用规则未命中的命令默认执行;命中的已知文件增删改、明确脚本/程序运行、系统/进程操作或网络写入先由本次运行主模型审核,模型明确判断操作在用户授权范围内时自动执行,否则再由用户审批。完全访问跳过两层审核。默认工作目录是用户主目录;终端改文件不进入修改记录,也不能从馆长回退。详细规则见 `WB-BLOCKER-001`。
|
||||
- AI 可通过 `web_search` / `kb_search` 检索;`web_search` 不得自动展开右侧列表。设置页正在「更新索引」时,`kb_search` / `kb_list_documents` 失败并提示正在更新知识库;「检查更新」不拦截。
|
||||
- AI 可通过 `web_search` / `kb_search` 检索;`web_search` 不得自动展开右侧列表。所有 KB 工具(含关键词检索与文档列表)统一要求向量模型已安装,调用时按需初始化本地模型;缺失、下载中、初始化或执行失败只返回工具错误,提示 AI 当前状态下停止调用 KB,不终止 run,不下载、不修复、不静默降级。设置页正在「更新索引」时返回错误;「检查更新」不拦截。模型可用时,空库、空范围和查询无命中正常返回空结果,不扩大显式范围。
|
||||
- `@` 参考资料不能替代「已创建并选中伙伴」这一发送前提。
|
||||
|
||||
## 6. 术语
|
||||
@@ -309,6 +309,7 @@ PPT 普通预览和审核需完整显示中文及全角标点,换行测量与
|
||||
- `WB-COMPOSER-001-AC2` 未选中伙伴或当前提交请求尚未返回时,不会重复触发同一次发送;普通 AI run 运行期间仍可提交新的追加消息。
|
||||
- `WB-COMPOSER-001-AC3` 已选中伙伴但发送按钮因前置条件不可执行时,hover 可见具体禁用原因。未创建或未选中伙伴时 Composer 为空态、不出现可发送按钮(见 `WB-CORE-001-AC2` / `WB-SESSION-001-AC1`),不适用本条 hover 文案。
|
||||
- `WB-COMPOSER-001-AC4` Composer 空输入占位含当前伙伴名;审批指导输入占位为「告诉{当前伙伴名}该怎么做」;改名后占位与后续 Brain 自称同步新名(可导致 LLM 前缀缓存 miss,本地数据不丢)。
|
||||
- `WB-COMPOSER-001-AC5` 未配置工作空间、未下载向量模型或模型初始化失败均不阻止 AI run 启动;普通聊天不准备或下载向量模型。
|
||||
|
||||
#### WB-COMPOSER-002 键盘行为
|
||||
|
||||
|
||||
+15
-13
@@ -154,7 +154,7 @@ Word、PPT、Excel、普通文本统一由主 run 的文件会话管理。工具
|
||||
- fixture 可用 `tests/fixtures/workspace`;运行时作为设置页文件地址 + 伙伴,不再「绑定工作区才能跑」。
|
||||
- AI backend 回归以生产 `createAiRunContext` 为唯一 runtime 入口,使用真实 store、apply service 和 embedding service;文档 worker 经 `AiRunResourceManager` 在 run 结束时释放,释放失败不得覆盖任务结果。
|
||||
- `mock-llm` 只固定 LLM 输出,其余全部真实;`live` 走真实 `LlmConfigStore` 与生产 LLM executor。
|
||||
- 进入 `human` 前必须完成 embedding preflight;KB fixture 走真实导入 / 索引 / 检索。所有支持文件前置「文件名锚点」内容块;Excel 只索引文件名、文档标题与工作表名等元数据。embedding 缺失时走真实生产下载链,不得用测试替身。
|
||||
- AI run 与 human harness 不准备 embedding;KB 工具调用时统一检查模型安装与本地初始化,不隐式下载或静默降级。需要 KB 的 fixture 显式通过真实下载 / 索引准备模型和资料;不使用 KB 的回归不准备模型。所有支持文件前置「文件名锚点」内容块;Excel 只索引文件名、文档标题与工作表名等元数据。
|
||||
- 不得使用旧式 fake API。默认 `testTimeout` / `hookTimeout` 为 `900_000`;少数重型 live case 可覆写更长超时。
|
||||
|
||||
### 3.2 `mock-llm` 与 `live` 设计口径
|
||||
@@ -789,21 +789,23 @@ Unit `docx-diff` 补重复内容匹配、跨 run 字符、Unicode 分段、大
|
||||
- How:脚本化 brain 决策调用真实 `kb_list_documents`;在工具结果写入、`context_budget` 已完成检查后模拟一条正在校验的提交,延迟确认入队;验证 Brain 入口等待校验并回到交接边界,再用第二个真实 graph run 捕获首轮 brain messages。
|
||||
- Oracle:第一轮状态为 `yielded` 且真实工具结果已写入 log;未确认入队前不会启动第二次 brain 决策;第二轮状态为 `completed`,历史中工具结果先于尾部追加 user;第二轮不重复调用中断前工具。
|
||||
|
||||
##### `ai-run-service-embedding-preflight`
|
||||
##### `ai-run-service-without-knowledge`
|
||||
|
||||
- 测试类型:`backend`
|
||||
- What:`AiRunService` 会在进入 `human` 前执行有界 embedding preflight,知识库语义检索可继续复用该基础设施。
|
||||
- Why:知识库 embedding 初始化应在模型可用时提前完成,但不能无限阻塞 AI run 启动。
|
||||
- How:通过 `AiRunService.getInstance().run(...)` 启动真实 run,在 preflight 成功后执行普通 DOCX literal 读取链路。
|
||||
- Oracle:preflight 被调用一次;任务完成;DOCX literal 搜索返回真实命中且不依赖 embedding。
|
||||
- 测试类型:`backend / mock-llm`
|
||||
- What:未配置文件地址且向量模型缺失或损坏时,主 run、子 run 和续接仍能完成聊天、真实文件读取。
|
||||
- Why:普通 AI 运行不能受知识库配置影响,也不能隐式下载模型。
|
||||
- How:隔离空模型目录,使用生产 AiRunService 和 scripted LLM 完成 DOCX literal 读取、子任务与续接;另在真实损坏模型目录下完成聊天。不调用 KB,不替换 embedding executor 或就绪检查。
|
||||
- Oracle:run 完成;真实文件读取命中;主子运行有 brain 事件;模型目录不变,embedding 保持 uninitialized;不产生模型下载。
|
||||
- 最新验证:2026-10-09 本轮通过;原始日志:`tests/logs/ai/human/mock-llm/ai-run-service-without-knowledge/20261009-104153-012-aa92ff73`。
|
||||
|
||||
##### `ai-run-service-embedding-preflight-failure`
|
||||
##### `ai-run-service-kb-error-continues`
|
||||
|
||||
- 测试类型:`backend`
|
||||
- What:embedding preflight 失败时,`AiRunService` 会在进入 `human` 前直接失败收口。
|
||||
- Why:基础服务明确失败时继续进入 human graph 会制造后续工具层噪音,应该在入口阶段明确失败;初始化卡住则由有界 preflight 超时保护。
|
||||
- How:注入会抛错的 `ensureEmbeddingReady`,启动一次真实 `AiRunService` run。
|
||||
- Oracle:run 状态为 `failed`,只返回统一的用户可见错误,不泄露底层 preflight 原文,事件流中没有任何 `human_step`。
|
||||
- 测试类型:`backend / mock-llm`
|
||||
- What:真实 KB 不可用只产生工具错误,后续文件工具和最终交付仍正常。
|
||||
- Why:知识库失败应交给 AI 决策,不能终止整个 run。
|
||||
- How:隔离空模型目录,生产 AiRunService 中 scripted LLM 先调用 kb_search,再读取真实文本并交付;捕获下一轮模型上下文。
|
||||
- Oracle:KB 结果 ok=false 且上下文含模型缺失原因及停止调用 KB 指引;真实文本读取成功;run completed,无 run_failed,目录无下载产物。
|
||||
- 最新验证:2026-10-09 本轮通过;原始日志:`tests/logs/ai/human/mock-llm/ai-run-service-kb-error-continues/20261009-104153-378-156b33dd`。
|
||||
|
||||
##### `learning-hand-mouth-smoke`
|
||||
|
||||
|
||||
@@ -22,7 +22,7 @@
|
||||
### 2.1 `foundation`
|
||||
|
||||
- 目标:守住知识库可回归的功能主链。
|
||||
- 约束:走生产 store / job / 扫描 / worker / 嵌入 executor;禁止假 executor 或假 `assertKbSearchAvailable`。索引速度 case 必须在独立 Node 进程里建真实 embedding session,并用操作系统线程数证明档位生效,不测 UI、不测墙钟 / CPU%。
|
||||
- 约束:走生产 store / job / 扫描 / worker / 嵌入 executor;禁止假 executor 或假 `assertKbAvailable`。索引速度 case 必须在独立 Node 进程里建真实 embedding session,并用操作系统线程数证明档位生效,不测 UI、不测墙钟 / CPU%。
|
||||
- 目录:`tests/case/knowledge/foundation/<case>.test.ts`
|
||||
|
||||
### 2.2 `optimization`
|
||||
@@ -82,8 +82,18 @@
|
||||
- 测试类型:`backend`
|
||||
- What:prepare 索引进行中时 `kb_search` / `kb_list_documents` 失败,并投递模型可读错误。
|
||||
- Why:更新索引期间检索必须明确失败,避免半库结果。
|
||||
- How:隔离 sqlite 写入 running prepare job,再走生产 `createAiRunContext` 调用两个工具,并检查 model-facing 投递文案。
|
||||
- Oracle:两工具抛出正在更新知识库错误;模型侧结果为 `Error: 正在更新知识库索引,请完成后再检索。`,对齐 `SET-WORKSPACE-004-AC9`、`SET-WEB-004-AC8`。
|
||||
- How:隔离 sqlite 写入文件 / 网页 running prepare job,再走生产 `createAiRunContext` 调用两个工具并检查投递文案;显式准备真实模型后,验证 running scan 与 prepare 的 succeeded / failed / cancelled 终态不拦截空库查询。
|
||||
- Oracle:两工具抛出正在更新知识库错误;模型侧结果包含正在更新知识库原因及当前状态下停止调用 KB 的指引,对齐 `SET-WORKSPACE-004-AC9`、`SET-WEB-004-AC8`。
|
||||
- 最新验证:2026-10-09 本轮通过;原始日志:`tests/logs/knowledge/foundation/kb-indexing-blocks-search/20261009-104411-354-11c9b53e`。
|
||||
|
||||
#### `kb-availability-contract`
|
||||
|
||||
- 测试类型:`backend`
|
||||
- What:所有 KB 工具统一检查模型安装 / 初始化,空库和空范围正常返回空结果;索引入口同样不隐式下载。
|
||||
- Why:模型缺失不能被关键词或列表绕过;空库不能误判为知识库故障。
|
||||
- How:独立模型目录中用真实 ModelService、EmbeddingService / Transformers executor、store 和 createAiRunContext;分别准备空目录、不完整下载目录、文件齐全但内容损坏的模型目录,调用两个 KB 工具与文件 / 网页索引入口。健康模型显式准备后验证空库、限定空范围、无文件地址的全局网页、删除与恢复。不替换执行器和就绪检查;Node 使用同一生产 executor in-process,Electron utilityProcess 由 Workbench E2E 验证。
|
||||
- Oracle:缺失 / 不完整模型在所有 strategy 和列表中返回错误且目录不变;损坏模型初始化错误;索引入口缺失模型失败且无资料写入;健康模型空库 / 空范围返回 items=[]、total=0;全局网页能检索;模型删除后下一次 KB 失败,恢复后可用。
|
||||
- 最新验证:2026-10-09 本轮通过;原始日志:`tests/logs/knowledge/foundation/kb-availability-contract/20261009-104115-815-12bf90e6`。
|
||||
|
||||
#### `index-speed-thread-budget`
|
||||
|
||||
|
||||
@@ -185,6 +185,15 @@ pnpm test:update:e2e
|
||||
- How:完成一次真实 AI run,读取圆圈可访问名称和 hover 提示。用量环分母为当前所选模型的 effectiveWindow(免费/推荐目录 `contextWindowTokens` × 0.8 再 cap 256k;自定义按 256k 算出)。
|
||||
- Oracle:圆圈展示已用信息,hover 展示已用百分比与 tokens、不展示剩余,对齐 `WB-COMPOSER-008`。
|
||||
|
||||
#### `chat-without-knowledge-model`
|
||||
|
||||
- 测试类型:`e2e / live`
|
||||
- What:未配置工作空间、未安装向量模型仍可发送并完成真实聊天。
|
||||
- Why:默认 E2E 模型准备不能掩盖首次使用的运行门禁。
|
||||
- How:通过既有 NCURATOR_MODELS_PATH 注入独立空模型目录,跳过 harness 默认模型预热;真实 UI 配聊天模型、创建伙伴并发送;只读 observer 收集运行事件、下载状态和 embedding runtime。
|
||||
- Oracle:真实 run completed;模型目录没有下载产物;无下载任务;embedding runtime uninitialized 且 pid 为空,对齐 WB-COMPOSER-001-AC5。
|
||||
- 最新验证:2026-10-09 本轮通过;原始日志:`tests/logs/workbench/conversation/composer/chat-without-knowledge-model/20261009-104143-882`。
|
||||
|
||||
#### `send-guard-and-preferences`
|
||||
|
||||
- 测试类型:`e2e`
|
||||
|
||||
@@ -7,7 +7,6 @@ import { releasePdfRunSession } from '../pdf/worker/pdfWorkerClient'
|
||||
import { terminalSessionManager } from './human/tools/terminal/TerminalSessionManager'
|
||||
import { createHumanGraphRunner } from './human/graph/humanGraph'
|
||||
import { createAiRunContext } from './runtime'
|
||||
import { runAiRuntimePreflight } from './runtimePreflight'
|
||||
import {
|
||||
createCancelledAiRunState,
|
||||
createCompletedAiRunState,
|
||||
@@ -75,7 +74,6 @@ export class AiRunService {
|
||||
}
|
||||
}
|
||||
try {
|
||||
await runAiRuntimePreflight(deps)
|
||||
const ctx = createAiRunContext(payload, {
|
||||
...deps,
|
||||
delegateToSubagent: (tasks, toolCallId) => dispatch(tasks, toolCallId)
|
||||
|
||||
@@ -163,7 +163,7 @@ const normalizeKbSearchItems = (data: JsonValue | undefined): JsonObject[] => {
|
||||
|
||||
export const renderKbModelError: ModelResultRenderer = ({ data }) => {
|
||||
const detail = isObject(data) ? readString(data.error) : ''
|
||||
return detail ? `Error: ${detail}` : 'Error: kb tool failed.'
|
||||
return `${detail ? `Error: ${detail}` : 'Error: kb tool failed.'}\n当前状态下不要继续调用 KB 工具;请告知用户原因,可继续使用其他可用能力。`
|
||||
}
|
||||
|
||||
export const renderKbSearchModelResult: ModelResultRenderer = ({ data }) => {
|
||||
|
||||
@@ -65,9 +65,7 @@ const buildKbResult = (input: {
|
||||
})()
|
||||
|
||||
const normalizedData =
|
||||
input.ok === false && input.error
|
||||
? ({ error: input.error } satisfies JsonObject)
|
||||
: data
|
||||
input.ok === false && input.error ? ({ error: input.error } satisfies JsonObject) : data
|
||||
|
||||
return {
|
||||
toolName: input.toolName,
|
||||
@@ -165,7 +163,7 @@ export const buildKbCapabilityTools = (input: {
|
||||
args: args as JsonObject,
|
||||
execute: async (session) => {
|
||||
ctx.checkCancelled()
|
||||
await ctx.assertKbSearchAvailable()
|
||||
await ctx.assertKbAvailable()
|
||||
const prepared = resolveSearchArgs(args)
|
||||
if (!prepared.queryText) {
|
||||
throw new Error('知识库搜索需要提供查询文本。')
|
||||
@@ -195,20 +193,7 @@ export const buildKbCapabilityTools = (input: {
|
||||
prepared.requestedDocumentIds || undefined
|
||||
)
|
||||
}
|
||||
let vec: Float32Array
|
||||
try {
|
||||
const embedding = await ctx.embeddingService.embed(prepared.queryText)
|
||||
vec = embedding.data
|
||||
} catch (error) {
|
||||
if (prepared.strategy !== 'auto') throw error
|
||||
return await ctx.documentsStore.ftsSearch(
|
||||
prepared.queryText,
|
||||
prepared.limit,
|
||||
prepared.filter,
|
||||
prepared.workspaceId,
|
||||
prepared.requestedDocumentIds || undefined
|
||||
)
|
||||
}
|
||||
const { data: vec } = await ctx.embeddingService.embed(prepared.queryText)
|
||||
if (prepared.strategy === 'semantic') {
|
||||
return await ctx.documentsStore.vectorSearch(
|
||||
vec,
|
||||
@@ -273,7 +258,7 @@ export const buildKbCapabilityTools = (input: {
|
||||
args: args as JsonObject,
|
||||
execute: async (session) => {
|
||||
ctx.checkCancelled()
|
||||
await ctx.assertKbSearchAvailable()
|
||||
await ctx.assertKbAvailable()
|
||||
const workspaceId = String(ctx.workspace?.workspaceId || '').trim() || undefined
|
||||
const sourceNameQuery = String(args.sourceNameQuery || '').trim() || undefined
|
||||
const page = Math.max(1, Number(args.page ?? 1))
|
||||
|
||||
@@ -201,10 +201,11 @@ export const createAiRunContext = (payload: AiRunStartPayload, deps: AiRunDeps):
|
||||
},
|
||||
documentsStore: deps.documentsStore,
|
||||
embeddingService: deps.embeddingService,
|
||||
assertKbSearchAvailable: async () => {
|
||||
assertKbAvailable: async () => {
|
||||
if (await deps.hasRunningKnowledgeIndexJob?.()) {
|
||||
throw new Error(KB_INDEXING_IN_PROGRESS_ERROR)
|
||||
}
|
||||
await deps.ensureEmbeddingReady()
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,37 +0,0 @@
|
||||
import type { AiRunDeps } from './types'
|
||||
|
||||
const DEFAULT_EMBEDDING_PREFLIGHT_TIMEOUT_MS = 15_000
|
||||
|
||||
const readEmbeddingPreflightTimeoutMs = (): number => {
|
||||
const raw = Number(process.env.NCURATOR_AI_EMBEDDING_PREFLIGHT_TIMEOUT_MS)
|
||||
if (!Number.isFinite(raw)) return DEFAULT_EMBEDDING_PREFLIGHT_TIMEOUT_MS
|
||||
return Math.max(0, Math.floor(raw))
|
||||
}
|
||||
|
||||
export const runAiRuntimePreflight = async (
|
||||
deps: Pick<AiRunDeps, 'ensureEmbeddingReady'>
|
||||
): Promise<void> => {
|
||||
const timeoutMs = readEmbeddingPreflightTimeoutMs()
|
||||
if (timeoutMs === 0) {
|
||||
await deps.ensureEmbeddingReady()
|
||||
return
|
||||
}
|
||||
|
||||
const ensureReadyPromise = deps.ensureEmbeddingReady()
|
||||
let timeoutHandle: ReturnType<typeof setTimeout> | undefined
|
||||
const timeoutPromise = new Promise<'timeout'>((resolve) => {
|
||||
timeoutHandle = setTimeout(() => resolve('timeout'), timeoutMs)
|
||||
})
|
||||
|
||||
const result = await Promise.race([
|
||||
ensureReadyPromise.then(() => 'ready' as const),
|
||||
timeoutPromise
|
||||
])
|
||||
if (timeoutHandle) clearTimeout(timeoutHandle)
|
||||
if (result === 'ready') return
|
||||
|
||||
ensureReadyPromise.catch((error) => {
|
||||
console.warn('[AI-RUN] embedding preflight completed after timeout with error:', error)
|
||||
})
|
||||
console.warn(`[AI-RUN] embedding preflight timed out after ${timeoutMs}ms; continuing run`)
|
||||
}
|
||||
@@ -450,7 +450,7 @@ export type AiRunContext = {
|
||||
}) => Promise<AiFileChangeRecord[]>
|
||||
documentsStore: DocumentsStore
|
||||
embeddingService: EmbeddingService
|
||||
assertKbSearchAvailable: () => Promise<void>
|
||||
assertKbAvailable: () => Promise<void>
|
||||
}
|
||||
|
||||
export type AiRunFailureMetadata = {
|
||||
@@ -475,7 +475,7 @@ export type KnowledgeRetrievalContext = Pick<
|
||||
| 'workspace'
|
||||
| 'documentsStore'
|
||||
| 'embeddingService'
|
||||
| 'assertKbSearchAvailable'
|
||||
| 'assertKbAvailable'
|
||||
>
|
||||
|
||||
export type DocxToolContext = AiRunContext
|
||||
@@ -144,6 +144,7 @@ export class ModelService {
|
||||
}
|
||||
|
||||
private isModelDownloadedAtPath(repoId: string, modelPath: string): boolean {
|
||||
if (fs.existsSync(path.join(modelPath, DOWNLOAD_STATE_FILE))) return false
|
||||
if (!fs.existsSync(modelPath) || !fs.statSync(modelPath).isDirectory()) {
|
||||
return false
|
||||
}
|
||||
@@ -343,9 +344,7 @@ export class ModelService {
|
||||
|
||||
private resolveSourceEndpoint(source: ModelSource): string {
|
||||
if (source === 'modelscope') {
|
||||
return (
|
||||
String(process.env[MODELSCOPE_ENDPOINT_ENV] || '').trim() || MODELSCOPE_ENDPOINT
|
||||
)
|
||||
return String(process.env[MODELSCOPE_ENDPOINT_ENV] || '').trim() || MODELSCOPE_ENDPOINT
|
||||
}
|
||||
return String(process.env[HF_ENDPOINT_ENV] || '').trim() || HF_ENDPOINT
|
||||
}
|
||||
|
||||
@@ -150,11 +150,13 @@ export class VectorModelDownloadService {
|
||||
}
|
||||
|
||||
try {
|
||||
if (repoId === this.deps.embeddingService.getModelName()) {
|
||||
await this.deps.embeddingService.close()
|
||||
}
|
||||
await this.deps.modelService.deleteModel(repoId)
|
||||
if (this.activeSnapshot?.repoId === repoId) {
|
||||
this.clearSnapshot()
|
||||
}
|
||||
await this.deps.embeddingService.initialize().catch(() => undefined)
|
||||
return { success: true }
|
||||
} catch (error: unknown) {
|
||||
return {
|
||||
@@ -178,10 +180,6 @@ export class VectorModelDownloadService {
|
||||
signal,
|
||||
source
|
||||
)
|
||||
.then(async (value) => {
|
||||
await this.deps.embeddingService.initialize().catch(() => undefined)
|
||||
return value
|
||||
})
|
||||
.catch((error: unknown) => ({
|
||||
success: false,
|
||||
error: getErrorMessage(error),
|
||||
|
||||
@@ -7,7 +7,14 @@ import type {
|
||||
SearchResult,
|
||||
WebSourceRecord
|
||||
} from '../../../types/store'
|
||||
import { asRecord, toNumber, toNumberArray, toOptionalNumber, toOptionalString, toString } from '../../../utils/decoder'
|
||||
import {
|
||||
asRecord,
|
||||
toNumber,
|
||||
toNumberArray,
|
||||
toOptionalNumber,
|
||||
toOptionalString,
|
||||
toString
|
||||
} from '../../../utils/decoder'
|
||||
import { LanceDbCore, LANCE_TABLES } from '../core/LanceDbCore'
|
||||
import type { SqliteCore } from '../sqlite/SqliteCore'
|
||||
import {
|
||||
@@ -258,9 +265,9 @@ export class DocumentsStore {
|
||||
public async getDocumentByFilePath(filePath: string): Promise<DocumentRecord | null> {
|
||||
const normalized = String(filePath || '').trim()
|
||||
if (!normalized) return null
|
||||
const row = this.sqlite.prepare('SELECT * FROM document WHERE file_path = ? LIMIT 1').get(normalized) as
|
||||
| DocumentRow
|
||||
| undefined
|
||||
const row = this.sqlite
|
||||
.prepare('SELECT * FROM document WHERE file_path = ? LIMIT 1')
|
||||
.get(normalized) as DocumentRow | undefined
|
||||
if (!row) return null
|
||||
const [document] = await this.attachWebSourceMetadata([toDocumentRecord(row)])
|
||||
return document || null
|
||||
@@ -348,7 +355,10 @@ export class DocumentsStore {
|
||||
for (let index = 0; index < chunks.length; index += 1) {
|
||||
const chunk = chunks[index]
|
||||
const textBytes = Buffer.byteLength(chunk.text || '', 'utf8')
|
||||
if (batch.length >= batchSize || (batch.length > 0 && batchTextBytes + textBytes > maxTextBytes)) {
|
||||
if (
|
||||
batch.length >= batchSize ||
|
||||
(batch.length > 0 && batchTextBytes + textBytes > maxTextBytes)
|
||||
) {
|
||||
await flush()
|
||||
}
|
||||
batch.push({
|
||||
@@ -379,6 +389,7 @@ export class DocumentsStore {
|
||||
const table = await this.vector.openTable(LANCE_TABLES.CHUNK)
|
||||
const query = table.query()
|
||||
const where = await this.buildChunkWhere({ workspaceId, sourceType, documentIds })
|
||||
if ((await table.countRows(where)) === 0) return []
|
||||
if (where) query.where(where)
|
||||
const results = await query.nearestTo(queryVector).distanceType('cosine').limit(limit).toArray()
|
||||
return results.map(toSearchResult)
|
||||
@@ -394,6 +405,7 @@ export class DocumentsStore {
|
||||
const table = await this.vector.openTable(LANCE_TABLES.CHUNK)
|
||||
const q = table.query()
|
||||
const where = await this.buildChunkWhere({ workspaceId, sourceType, documentIds })
|
||||
if ((await table.countRows(where)) === 0) return []
|
||||
if (where) q.where(where)
|
||||
const results = await q.fullTextSearch(queryText).limit(limit).toArray()
|
||||
return results.map(toSearchResult)
|
||||
@@ -407,11 +419,12 @@ export class DocumentsStore {
|
||||
workspaceId?: string,
|
||||
documentIds?: string[]
|
||||
): Promise<SearchResult[]> {
|
||||
const reranker = await this.getRRFReranker()
|
||||
const table = await this.vector.openTable(LANCE_TABLES.CHUNK)
|
||||
const q = table.query()
|
||||
const where = await this.buildChunkWhere({ workspaceId, sourceType, documentIds })
|
||||
if ((await table.countRows(where)) === 0) return []
|
||||
if (where) q.where(where)
|
||||
const reranker = await this.getRRFReranker()
|
||||
const results = await q
|
||||
.fullTextSearch(queryText)
|
||||
.nearestTo(queryVector)
|
||||
@@ -469,7 +482,9 @@ export class DocumentsStore {
|
||||
.get(sqlParams) as { count?: unknown }
|
||||
const skip = Math.max(0, (page - 1) * pageSize)
|
||||
const rows = this.sqlite
|
||||
.prepare(`SELECT * FROM document WHERE ${where} ORDER BY created_at DESC LIMIT @limit OFFSET @offset`)
|
||||
.prepare(
|
||||
`SELECT * FROM document WHERE ${where} ORDER BY created_at DESC LIMIT @limit OFFSET @offset`
|
||||
)
|
||||
.all({ ...sqlParams, limit: pageSize, offset: skip }) as DocumentRow[]
|
||||
const items = await this.attachWebSourceMetadata(rows.map(toDocumentRecord))
|
||||
return { items, total: toRequiredNumber(totalRow.count) }
|
||||
@@ -494,7 +509,9 @@ export class DocumentsStore {
|
||||
private async getDocumentsByIds(ids: string[]): Promise<DocumentRecord[]> {
|
||||
const params: Record<string, string | number | null> = {}
|
||||
const where = buildWhere([buildInClause('id', ids, params, 'document_id')])
|
||||
const rows = this.sqlite.prepare(`SELECT * FROM document WHERE ${where}`).all(params) as DocumentRow[]
|
||||
const rows = this.sqlite
|
||||
.prepare(`SELECT * FROM document WHERE ${where}`)
|
||||
.all(params) as DocumentRow[]
|
||||
return await this.attachWebSourceMetadata(rows.map(toDocumentRecord))
|
||||
}
|
||||
|
||||
@@ -505,7 +522,9 @@ export class DocumentsStore {
|
||||
if (webDocumentIds.length === 0) return documents
|
||||
const params: Record<string, string | number | null> = {}
|
||||
const where = buildWhere([buildInClause('document_id', webDocumentIds, params, 'document_id')])
|
||||
const rows = this.sqlite.prepare(`SELECT * FROM web_source WHERE ${where}`).all(params) as WebSourceRecord[]
|
||||
const rows = this.sqlite
|
||||
.prepare(`SELECT * FROM web_source WHERE ${where}`)
|
||||
.all(params) as WebSourceRecord[]
|
||||
const webSourceMap = new Map(
|
||||
rows.map((row) => {
|
||||
const source = row as WebSourceRecord
|
||||
|
||||
@@ -87,6 +87,7 @@ export class EmbeddingService {
|
||||
}
|
||||
|
||||
public async close(): Promise<void> {
|
||||
await this.initializationPromise?.catch(() => undefined)
|
||||
await this.executor.close?.()
|
||||
this.status = 'uninitialized'
|
||||
}
|
||||
|
||||
@@ -1,21 +1,18 @@
|
||||
import type { ModelDownloadProgressReporter } from '../model/ModelService'
|
||||
import { ModelService } from '../model/ModelService'
|
||||
import { EmbeddingService } from './EmbeddingService'
|
||||
|
||||
export const EMBEDDING_MODEL_NOT_INSTALLED_ERROR =
|
||||
'向量模型尚未安装或下载未完成,请在设置页完成模型下载后再使用知识库。'
|
||||
|
||||
export const ensureEmbeddingReady = async (input: {
|
||||
embeddingService: EmbeddingService
|
||||
modelService?: ModelService
|
||||
reporter?: ModelDownloadProgressReporter
|
||||
}): Promise<void> => {
|
||||
if (input.embeddingService.getStatus() === 'ready') {
|
||||
return
|
||||
}
|
||||
|
||||
const modelService = input.modelService || ModelService.getInstance()
|
||||
const repoId = input.embeddingService.getModelName()
|
||||
|
||||
if (!modelService.isModelDownloaded(repoId)) {
|
||||
await modelService.downloadModel(repoId, input.reporter)
|
||||
throw new Error(EMBEDDING_MODEL_NOT_INSTALLED_ERROR)
|
||||
}
|
||||
|
||||
await input.embeddingService.initialize()
|
||||
@@ -24,7 +21,5 @@ export const ensureEmbeddingReady = async (input: {
|
||||
return
|
||||
}
|
||||
|
||||
throw new Error(
|
||||
`embedding bootstrap failed after ensuring model download; model=${repoId}; status=${input.embeddingService.getStatus()}`
|
||||
)
|
||||
throw new Error(`向量模型未就绪:${repoId} (${input.embeddingService.getStatus()})`)
|
||||
}
|
||||
@@ -2,6 +2,7 @@ import path from 'node:path'
|
||||
import { v4 as uuidv4 } from 'uuid'
|
||||
import type { DocumentsStore } from '../storage/domains/DocumentsStore'
|
||||
import type { EmbeddingService } from '../vector/EmbeddingService'
|
||||
import { ensureEmbeddingReady } from '../vector/ensureEmbeddingReady'
|
||||
import { isSupportedLocalFilePath } from '../ingestion/loader'
|
||||
import type {
|
||||
WorkspaceKnowledgeChunkInput,
|
||||
@@ -77,6 +78,8 @@ export class WorkspaceKnowledgePrepareService {
|
||||
if (!isSupportedLocalFilePath(input.filePath)) {
|
||||
throw new Error(`Unsupported file type: ${input.filePath}`)
|
||||
}
|
||||
await ensureEmbeddingReady({ embeddingService: this.deps.embeddingService })
|
||||
this.throwIfAborted(input.signal)
|
||||
|
||||
const stopAbort = this.listenForAbort(input.signal, () => {
|
||||
void this.abortActivePrepare()
|
||||
@@ -146,6 +149,8 @@ export class WorkspaceKnowledgePrepareService {
|
||||
public async indexWebText(
|
||||
input: WorkspaceKnowledgeWebInput
|
||||
): Promise<WorkspaceKnowledgeIndexResult> {
|
||||
this.throwIfAborted(input.signal)
|
||||
await ensureEmbeddingReady({ embeddingService: this.deps.embeddingService })
|
||||
this.throwIfAborted(input.signal)
|
||||
const stopAbort = this.listenForAbort(input.signal, () => {
|
||||
void this.abortActivePrepare()
|
||||
@@ -254,14 +259,6 @@ export class WorkspaceKnowledgePrepareService {
|
||||
vectors: []
|
||||
}
|
||||
} else {
|
||||
if (this.deps.embeddingService.getStatus() !== 'ready') {
|
||||
await this.deps.embeddingService.initialize()
|
||||
}
|
||||
if (this.deps.embeddingService.getStatus() !== 'ready') {
|
||||
throw new Error(
|
||||
`EmbeddingService is not ready. Current status: ${this.deps.embeddingService.getStatus()}`
|
||||
)
|
||||
}
|
||||
vectorSnapshot = {
|
||||
contentHash: preparedSource.contentHash,
|
||||
fileExt: preparedSource.fileExt,
|
||||
@@ -402,8 +399,7 @@ export class WorkspaceKnowledgePrepareService {
|
||||
}))
|
||||
}
|
||||
} catch (error) {
|
||||
const cancelled =
|
||||
isIndexPrepareCancelledError(error) || Boolean(input.signal?.aborted)
|
||||
const cancelled = isIndexPrepareCancelledError(error) || Boolean(input.signal?.aborted)
|
||||
if (wroteChunks && !activated) {
|
||||
await this.deps.documentsStore
|
||||
.deleteChunksByGeneration({
|
||||
|
||||
@@ -1,225 +0,0 @@
|
||||
import path from 'node:path'
|
||||
import { describe, it } from 'vitest'
|
||||
import { AiRunService } from '../../../../../src/main/services/ai/AiRunService'
|
||||
import { AiFileChangeService } from '../../../../../src/main/services/ai/fileChanges/AiFileChangeService'
|
||||
import { runBackendCase } from '../../../../harness/backend/run-backend-case'
|
||||
import { createScriptedStructuredOutputDriver } from '../../../../harness/backend/scriptedStructuredOutputDriver'
|
||||
import { writeTestDocxFile } from '../../../../harness/backend/aiToolHarness'
|
||||
import type { DocxSearchResult } from '../../tool/docx/testUtils'
|
||||
import {
|
||||
buildCompleteTaskDecision,
|
||||
buildLearnSkillsDecision,
|
||||
buildToolDecision,
|
||||
filterAiRunEventsByType,
|
||||
readLatestToolResultDataFromEvents
|
||||
} from '../testUtils'
|
||||
|
||||
describe('ai human mock-llm backend', () => {
|
||||
it('ai-run-service-embedding-preflight', async () => {
|
||||
await runBackendCase({
|
||||
caseId: 'ai-run-service-embedding-preflight',
|
||||
harnessMode: 'mock-llm',
|
||||
domain: 'ai',
|
||||
suite: 'human',
|
||||
groups: ['mock-llm'],
|
||||
description:
|
||||
'验证 AiRunService 会在进入 human 前完成 embedding preflight,并支撑后续 docx 读取。',
|
||||
fixtures: [],
|
||||
runMode: 'single',
|
||||
execute: async (runtime) => {
|
||||
await runtime.chatStore.savePartner({
|
||||
id: 'embedding-preflight-session',
|
||||
name: 'embedding preflight',
|
||||
avatar_shape: 'circle',
|
||||
avatar_color: 'blue',
|
||||
created_at: Date.now(),
|
||||
updated_at: Date.now(),
|
||||
last_viewed_at: Date.now(),
|
||||
pinned: false,
|
||||
pinned_at: null
|
||||
})
|
||||
await writeTestDocxFile(runtime.workspaceRoot, 'literal-docx.docx', [
|
||||
{
|
||||
type: 'paragraph',
|
||||
text: '反走样是在采样不足时出现的锯齿现象。'
|
||||
}
|
||||
])
|
||||
|
||||
let ensureReadyCalls = 0
|
||||
const initialEmbeddingStatus = runtime.embeddingService.getStatus()
|
||||
const aiFileChangeService = new AiFileChangeService(runtime.storageService.aiFileChange)
|
||||
const result = await AiRunService.getInstance().run(
|
||||
{
|
||||
runId: runtime.runId,
|
||||
partnerId: 'embedding-preflight-session',
|
||||
input: '请读取 docx,找到反走样相关内容。',
|
||||
mainModel: {
|
||||
configId: 'mock-llm-scripted',
|
||||
modelName: 'scripted-llm'
|
||||
},
|
||||
partnerName: '测试伙伴',
|
||||
workspaceRoots: [runtime.workspaceRoot]
|
||||
},
|
||||
{
|
||||
llmStore: runtime.llmStore,
|
||||
partnerStore: runtime.chatStore,
|
||||
chatMessageLogStore: runtime.storageService.chatMessageLog,
|
||||
documentsStore: runtime.documentsStore,
|
||||
embeddingService: runtime.embeddingService,
|
||||
ensureEmbeddingReady: async () => {
|
||||
ensureReadyCalls += 1
|
||||
await runtime.ensureEmbeddingReady()
|
||||
},
|
||||
docxApplyService: runtime.docxApplyService,
|
||||
saveFileChangeRecord: async (record) => {
|
||||
await aiFileChangeService.saveRecord(record)
|
||||
},
|
||||
listFileChangeRecords: async (request) =>
|
||||
(await aiFileChangeService.listRecords(request)).items,
|
||||
isCancelled: () => false,
|
||||
sendEvent: (event) => runtime.eventCollector.add(event),
|
||||
recordLlmIo: (record) => runtime.eventCollector.addLlmIo(record),
|
||||
requestApprovalBatch: async () => ({ accepted: true, decisions: [] }),
|
||||
llmDriver: createScriptedStructuredOutputDriver({
|
||||
'human.brain': [
|
||||
{
|
||||
output: buildLearnSkillsDecision({
|
||||
skillIds: ['docx-read'],
|
||||
summary: '先学习 docx-read'
|
||||
})
|
||||
},
|
||||
{
|
||||
output: buildToolDecision({
|
||||
toolName: 'docx_search',
|
||||
args: {
|
||||
sourcePath: path.join(runtime.workspaceRoot, 'literal-docx.docx'),
|
||||
query: '锯齿现象',
|
||||
mode: 'literal'
|
||||
},
|
||||
summary: '搜索完整原文'
|
||||
})
|
||||
},
|
||||
{
|
||||
output: buildCompleteTaskDecision({
|
||||
summary: 'embedding preflight 已验证。',
|
||||
deliveryMarkdown: '# Embedding Preflight\n\ndocx 读取链路已可用。'
|
||||
})
|
||||
}
|
||||
]
|
||||
})
|
||||
}
|
||||
)
|
||||
|
||||
return {
|
||||
result,
|
||||
ensureReadyCalls,
|
||||
initialEmbeddingStatus,
|
||||
events: runtime.eventCollector.snapshot()
|
||||
}
|
||||
},
|
||||
assert: async (result, runtime) => {
|
||||
const docxSearchResult = readLatestToolResultDataFromEvents<DocxSearchResult>(
|
||||
result.events,
|
||||
'docx_search'
|
||||
)
|
||||
|
||||
runtime.assertThat(
|
||||
result.initialEmbeddingStatus === 'uninitialized' &&
|
||||
result.ensureReadyCalls === 1 &&
|
||||
result.result.status === 'completed' &&
|
||||
docxSearchResult.matches.length === 1 &&
|
||||
docxSearchResult.matches[0]?.matchText === '锯齿现象',
|
||||
{
|
||||
name: 'ai-run-service-preflights-embedding-before-human',
|
||||
message:
|
||||
'AiRunService 应在进入 human 前完成 embedding preflight,DOCX literal 搜索不依赖 embedding。',
|
||||
data: {
|
||||
initialEmbeddingStatus: result.initialEmbeddingStatus,
|
||||
ensureReadyCalls: result.ensureReadyCalls,
|
||||
docxSearchResult,
|
||||
finalStatus: result.result.status
|
||||
}
|
||||
}
|
||||
)
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
it('ai-run-service-embedding-preflight-failure', async () => {
|
||||
await runBackendCase({
|
||||
caseId: 'ai-run-service-embedding-preflight-failure',
|
||||
harnessMode: 'mock-llm',
|
||||
domain: 'ai',
|
||||
suite: 'human',
|
||||
groups: ['mock-llm'],
|
||||
description: '验证 embedding preflight 失败时,AiRunService 会在进入 human 前直接失败收口。',
|
||||
fixtures: [],
|
||||
runMode: 'single',
|
||||
execute: async (runtime) => {
|
||||
const aiFileChangeService = new AiFileChangeService(runtime.storageService.aiFileChange)
|
||||
const result = await AiRunService.getInstance().run(
|
||||
{
|
||||
runId: runtime.runId,
|
||||
partnerId: 'embedding-preflight-failure-session',
|
||||
input: '这轮不应该进入 human。',
|
||||
mainModel: {
|
||||
configId: 'mock-llm-scripted',
|
||||
modelName: 'scripted-llm'
|
||||
},
|
||||
partnerName: '测试伙伴',
|
||||
workspaceRoots: [runtime.workspaceRoot]
|
||||
},
|
||||
{
|
||||
llmStore: runtime.llmStore,
|
||||
partnerStore: runtime.chatStore,
|
||||
chatMessageLogStore: runtime.storageService.chatMessageLog,
|
||||
documentsStore: runtime.documentsStore,
|
||||
embeddingService: runtime.embeddingService,
|
||||
ensureEmbeddingReady: async () => {
|
||||
throw new Error('embedding preflight failed for test')
|
||||
},
|
||||
docxApplyService: runtime.docxApplyService,
|
||||
saveFileChangeRecord: async (record) => {
|
||||
await aiFileChangeService.saveRecord(record)
|
||||
},
|
||||
listFileChangeRecords: async (request) =>
|
||||
(await aiFileChangeService.listRecords(request)).items,
|
||||
isCancelled: () => false,
|
||||
sendEvent: (event) => runtime.eventCollector.add(event),
|
||||
recordLlmIo: (record) => runtime.eventCollector.addLlmIo(record),
|
||||
requestApprovalBatch: async () => ({ accepted: true, decisions: [] }),
|
||||
llmDriver: createScriptedStructuredOutputDriver({})
|
||||
}
|
||||
)
|
||||
|
||||
return {
|
||||
result,
|
||||
events: runtime.eventCollector.snapshot()
|
||||
}
|
||||
},
|
||||
assert: async (result, runtime) => {
|
||||
const humanSteps = filterAiRunEventsByType(result.events, 'human_step')
|
||||
|
||||
runtime.assertThat(
|
||||
result.result.status === 'failed' &&
|
||||
result.result.failureCode === 'unexpected_error' &&
|
||||
result.result.error === '运行失败,请重试' &&
|
||||
result.result.failureRawMessage === 'embedding preflight failed for test' &&
|
||||
humanSteps.length === 0,
|
||||
{
|
||||
name: 'ai-run-service-aborts-before-human-on-preflight-failure',
|
||||
message:
|
||||
'embedding preflight 失败时,run 应直接失败、不进入 human stage,并保留短标题与原文。',
|
||||
data: {
|
||||
status: result.result.status,
|
||||
failureCode: result.result.failureCode,
|
||||
error: result.result.error,
|
||||
failureRawMessage: result.result.failureRawMessage,
|
||||
humanStepCount: humanSteps.length
|
||||
}
|
||||
}
|
||||
)
|
||||
}
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,187 @@
|
||||
import fs from 'node:fs/promises'
|
||||
import path from 'node:path'
|
||||
import { describe, it, expect } from 'vitest'
|
||||
import { runBackendCase } from '../../../../harness/backend/run-backend-case'
|
||||
import { createAiServiceTestRunner } from '../../../../harness/backend/aiServiceHarness'
|
||||
import {
|
||||
withIsolatedModels,
|
||||
writeCorruptEmbeddingModel
|
||||
} from '../../../../harness/backend/modelFixture'
|
||||
import { writeTestDocxFile } from '../../../../harness/backend/aiToolHarness'
|
||||
import {
|
||||
buildCompleteTaskDecision,
|
||||
buildLearnSkillsDecision,
|
||||
buildToolDecision,
|
||||
readLatestToolResultDataFromEvents
|
||||
} from '../testUtils'
|
||||
import type { ScriptedLlmStep } from '../../../../harness/backend/test-types'
|
||||
import { EMBEDDING_MODEL_NOT_INSTALLED_ERROR } from '../../../../../src/main/services/vector/ensureEmbeddingReady'
|
||||
|
||||
describe('AI runs independent of knowledge configuration', () => {
|
||||
it('ai-run-service-without-knowledge', async () => {
|
||||
await withIsolatedModels(async (modelsPath) => {
|
||||
await runBackendCase({
|
||||
caseId: 'ai-run-service-without-knowledge',
|
||||
domain: 'ai',
|
||||
suite: 'human',
|
||||
groups: ['mock-llm'],
|
||||
harnessMode: 'mock-llm',
|
||||
fixtures: [],
|
||||
runMode: 'single',
|
||||
description: '模型缺失或损坏、无工作地址仍能读取真实文件、派发子任务及续接。',
|
||||
assert: async () => {},
|
||||
execute: async (runtime) => {
|
||||
const sourcePath = path.join(runtime.workspaceRoot, 'local.txt')
|
||||
await fs.writeFile(sourcePath, 'LOCAL_READ_EVIDENCE')
|
||||
await writeTestDocxFile(runtime.workspaceRoot, 'literal.docx', [
|
||||
{ type: 'paragraph', text: '反走样减少锯齿现象。' }
|
||||
])
|
||||
let parentCalls = 0
|
||||
let childCalls = 0
|
||||
let childReadConfirmed = false
|
||||
const script: ScriptedLlmStep[] = Array.from({ length: 10 }, () => (call) => {
|
||||
if (/用户诉求:\nCHILD_LOCAL_READ/.test(call.user)) {
|
||||
childCalls += 1
|
||||
if (childCalls > 1) childReadConfirmed = call.user.includes('LOCAL_READ_EVIDENCE')
|
||||
return childCalls === 1
|
||||
? buildToolDecision({
|
||||
toolName: 'text_file_read',
|
||||
args: { sourcePath },
|
||||
summary: '读取明确指定的文件'
|
||||
})
|
||||
: buildCompleteTaskDecision({ summary: '子任务已读到真实文件' })
|
||||
}
|
||||
parentCalls += 1
|
||||
if (parentCalls === 1)
|
||||
return buildLearnSkillsDecision({
|
||||
skillIds: ['docx-read'],
|
||||
summary: '学习原文读取能力'
|
||||
})
|
||||
if (parentCalls === 2)
|
||||
return buildToolDecision({
|
||||
toolName: 'docx_search',
|
||||
args: {
|
||||
sourcePath: path.join(runtime.workspaceRoot, 'literal.docx'),
|
||||
query: '锯齿现象',
|
||||
mode: 'literal'
|
||||
},
|
||||
summary: '读取 DOCX 原文'
|
||||
})
|
||||
if (parentCalls === 3)
|
||||
return buildToolDecision({
|
||||
toolName: 'delegate_to_subagent',
|
||||
args: { tasks: [{ name: '读取', task: 'CHILD_LOCAL_READ 读取指定本地文本' }] },
|
||||
summary: '派发文件读取子任务'
|
||||
})
|
||||
return buildCompleteTaskDecision({ summary: '读取完成' })
|
||||
})
|
||||
const runner = await createAiServiceTestRunner(runtime, {
|
||||
bindWorkspace: false,
|
||||
partnerId: 'knowledge-independent',
|
||||
llmScript: { 'human.brain': script }
|
||||
})
|
||||
const output = await runner.run()
|
||||
expect(output.status).toBe('completed')
|
||||
expect(
|
||||
readLatestToolResultDataFromEvents<{ totalMatchCount: number }>(
|
||||
runner.events,
|
||||
'docx_search'
|
||||
).totalMatchCount
|
||||
).toBeGreaterThan(0)
|
||||
expect(childReadConfirmed).toBe(true)
|
||||
expect(
|
||||
runner.events.some(
|
||||
(e) => e.type === 'subagent_updated' && e.update.status === 'completed'
|
||||
)
|
||||
).toBe(true)
|
||||
expect(childCalls).toBeGreaterThan(0)
|
||||
expect(runtime.embeddingService.getStatus()).toBe('uninitialized')
|
||||
expect(await fs.readdir(modelsPath)).toEqual([])
|
||||
|
||||
await writeCorruptEmbeddingModel(modelsPath)
|
||||
const before = await fs.readdir(modelsPath, { recursive: true })
|
||||
const resumed = await createAiServiceTestRunner(runtime, {
|
||||
bindWorkspace: false,
|
||||
partnerId: 'knowledge-independent',
|
||||
runId: `${runtime.runId}-continue`,
|
||||
userInput: '继续确认刚才的结果',
|
||||
llmScript: {
|
||||
'human.brain': [{ output: buildCompleteTaskDecision({ summary: '续接完成' }) }]
|
||||
}
|
||||
})
|
||||
expect((await resumed.run()).status).toBe('completed')
|
||||
expect(runtime.embeddingService.getStatus()).toBe('uninitialized')
|
||||
expect(await fs.readdir(modelsPath, { recursive: true })).toEqual(before)
|
||||
runtime.assertThat(true, {
|
||||
name: 'runs-without-embedding',
|
||||
message: '主运行、子任务与续接无需 embedding,真实文件读取成功且模型目录不变。'
|
||||
})
|
||||
}
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
it('ai-run-service-kb-error-continues', async () => {
|
||||
await withIsolatedModels(async (modelsPath) => {
|
||||
await runBackendCase({
|
||||
caseId: 'ai-run-service-kb-error-continues',
|
||||
domain: 'ai',
|
||||
suite: 'human',
|
||||
groups: ['mock-llm'],
|
||||
harnessMode: 'mock-llm',
|
||||
fixtures: [],
|
||||
runMode: 'single',
|
||||
description: '真实 KB 模型缺失错误进入模型上下文,后续真实文件读取与交付继续。',
|
||||
assert: async () => {},
|
||||
execute: async (runtime) => {
|
||||
const sourcePath = path.join(runtime.workspaceRoot, 'source.txt')
|
||||
await fs.writeFile(sourcePath, 'KB_FAILURE_CONTINUED')
|
||||
const runner = await createAiServiceTestRunner(runtime, {
|
||||
bindWorkspace: false,
|
||||
llmScript: {
|
||||
'human.brain': [
|
||||
{
|
||||
output: buildToolDecision({
|
||||
toolName: 'kb_search',
|
||||
args: { queryText: '资料' },
|
||||
summary: '查询资料'
|
||||
})
|
||||
},
|
||||
(call) => {
|
||||
expect(call.user).toContain(EMBEDDING_MODEL_NOT_INSTALLED_ERROR)
|
||||
expect(call.user).toContain('不要继续调用 KB')
|
||||
return buildToolDecision({
|
||||
toolName: 'text_file_read',
|
||||
args: { sourcePath },
|
||||
summary: '读取明确指定的文件'
|
||||
})
|
||||
},
|
||||
{ output: buildCompleteTaskDecision({ summary: '已读取文件,知识库暂不可用' }) }
|
||||
]
|
||||
}
|
||||
})
|
||||
expect((await runner.run()).status).toBe('completed')
|
||||
expect(
|
||||
runner.events.some(
|
||||
(e) =>
|
||||
e.type === 'tool_call_result' &&
|
||||
e.toolName === 'kb_search' &&
|
||||
e.result?.ok === false
|
||||
)
|
||||
).toBe(true)
|
||||
expect(
|
||||
readLatestToolResultDataFromEvents<{ content: string }>(runner.events, 'text_file_read')
|
||||
.content
|
||||
).toContain('KB_FAILURE_CONTINUED')
|
||||
expect(runner.events.some((e) => e.type === 'run_failed')).toBe(false)
|
||||
expect(runtime.embeddingService.getStatus()).toBe('uninitialized')
|
||||
expect(await fs.readdir(modelsPath)).toEqual([])
|
||||
runtime.assertThat(true, {
|
||||
name: 'kb-error-does-not-fail-run',
|
||||
message: 'KB 错误已反馈给模型,文件读取与 run 完成,无隐式下载。'
|
||||
})
|
||||
}
|
||||
})
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -21,6 +21,7 @@ describe('ai human mock-llm backend', () => {
|
||||
fixtures: [],
|
||||
runMode: 'single',
|
||||
execute: async (runtime) => {
|
||||
await runtime.prepareEmbeddingModel()
|
||||
const blockedPath = path.join(runtime.workspaceRoot, 'blocked-dir')
|
||||
const runner = createHumanTestRunner(runtime, {
|
||||
approvalPolicy: 'deny-medium-high',
|
||||
|
||||
@@ -15,6 +15,7 @@ describe('ai human mock-llm backend', () => {
|
||||
fixtures: [],
|
||||
runMode: 'single',
|
||||
execute: async (runtime) => {
|
||||
await runtime.prepareEmbeddingModel()
|
||||
const runner = createHumanTestRunner(runtime, {
|
||||
structuredOutputScript: {
|
||||
'human.brain': [
|
||||
@@ -80,6 +81,7 @@ describe('ai human mock-llm backend', () => {
|
||||
fixtures: [],
|
||||
runMode: 'single',
|
||||
execute: async (runtime) => {
|
||||
await runtime.prepareEmbeddingModel()
|
||||
const runner = createHumanTestRunner(runtime, {
|
||||
structuredOutputScript: {
|
||||
'human.brain': [
|
||||
|
||||
@@ -18,6 +18,7 @@ describe('ai human mock-llm backend', () => {
|
||||
fixtures: [],
|
||||
runMode: 'single',
|
||||
execute: async (runtime) => {
|
||||
await runtime.prepareEmbeddingModel()
|
||||
let hasQueuedMessage = false
|
||||
let queueSubmissionStarted = false
|
||||
let toolResultRecorded = false
|
||||
@@ -126,7 +127,7 @@ describe('ai human mock-llm backend', () => {
|
||||
const toolResults = filterAiRunEventsByType(result.firstEvents, 'tool_call_result')
|
||||
const toolLogEntries = result.firstLog.filter((record) => record.entry.kind === 'tool')
|
||||
runtime.assertThat(
|
||||
result.firstOutput.status === 'yielded' &&
|
||||
result.firstOutput.status === 'yielded' &&
|
||||
result.firstBrainCallCount === 1 &&
|
||||
toolResults.some((event) => event.toolName === 'kb_list_documents') &&
|
||||
toolLogEntries.some(
|
||||
|
||||
@@ -15,6 +15,7 @@ describe('ai human mock-llm backend', () => {
|
||||
fixtures: [],
|
||||
runMode: 'single',
|
||||
execute: async (runtime) => {
|
||||
await runtime.prepareEmbeddingModel()
|
||||
await runtime.seedKb([
|
||||
{
|
||||
id: 'doc-1',
|
||||
|
||||
@@ -0,0 +1,163 @@
|
||||
import fs from 'node:fs/promises'
|
||||
import path from 'node:path'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { runBackendCase } from '../../../harness/backend/run-backend-case'
|
||||
import { createRetrievalToolHarness } from '../../../harness/backend/aiToolHarness'
|
||||
import {
|
||||
withIsolatedModels,
|
||||
copyInstalledEmbeddingModel,
|
||||
writeCorruptEmbeddingModel
|
||||
} from '../../../harness/backend/modelFixture'
|
||||
import { ModelService } from '../../../../src/main/services/model/ModelService'
|
||||
import { VectorModelDownloadService } from '../../../../src/main/services/model/VectorModelDownloadService'
|
||||
import { DEFAULT_EMBEDDING_MODEL_NAME } from '../../../../src/main/services/vector/EmbeddingService'
|
||||
import { EMBEDDING_MODEL_NOT_INSTALLED_ERROR } from '../../../../src/main/services/vector/ensureEmbeddingReady'
|
||||
|
||||
describe('knowledge availability through real services', () => {
|
||||
it('kb-availability-contract', async () => {
|
||||
await withIsolatedModels(async (modelsPath, sharedModelsPath) => {
|
||||
await runBackendCase({
|
||||
caseId: 'kb-availability-contract',
|
||||
domain: 'knowledge',
|
||||
suite: 'foundation',
|
||||
harnessMode: 'tool-real',
|
||||
fixtures: [],
|
||||
runMode: 'single',
|
||||
description: '真实模型安装门禁、初始化失败、空库、全局网页、删除恢复及索引入口。',
|
||||
assert: async () => {},
|
||||
execute: async (runtime) => {
|
||||
const harness = createRetrievalToolHarness(runtime, { bindWorkspace: false })
|
||||
const modelPath = path.join(modelsPath, DEFAULT_EMBEDDING_MODEL_NAME)
|
||||
const modelService = ModelService.getInstance()
|
||||
const assertAllUnavailable = async () => {
|
||||
const before = await fs.readdir(modelsPath, { recursive: true })
|
||||
for (const strategy of ['auto', 'semantic', 'keyword']) {
|
||||
await expect(
|
||||
harness.invoke('kb_search', { queryText: '资料', strategy })
|
||||
).rejects.toThrow(EMBEDDING_MODEL_NOT_INSTALLED_ERROR)
|
||||
}
|
||||
await expect(harness.invoke('kb_list_documents', {})).rejects.toThrow(
|
||||
EMBEDDING_MODEL_NOT_INSTALLED_ERROR
|
||||
)
|
||||
expect(runtime.embeddingService.getStatus()).toBe('uninitialized')
|
||||
expect(await fs.readdir(modelsPath, { recursive: true })).toEqual(before)
|
||||
}
|
||||
await assertAllUnavailable()
|
||||
await fs.mkdir(modelPath, { recursive: true })
|
||||
await fs.writeFile(path.join(modelPath, 'config.json'), '{}')
|
||||
await assertAllUnavailable()
|
||||
|
||||
const sourcePath = path.join(runtime.workspaceRoot, 'source.txt')
|
||||
await fs.writeFile(sourcePath, '索引输入')
|
||||
await expect(
|
||||
runtime.workspaceKnowledgePrepareService.indexLocalFile({
|
||||
workspaceId: 'source',
|
||||
documentId: 'source',
|
||||
documentName: 'source.txt',
|
||||
filePath: sourcePath
|
||||
})
|
||||
).rejects.toThrow(EMBEDDING_MODEL_NOT_INSTALLED_ERROR)
|
||||
await expect(
|
||||
runtime.workspaceKnowledgePrepareService.indexWebText({
|
||||
sourceType: 'web',
|
||||
workspaceId: '',
|
||||
documentId: 'web-source',
|
||||
documentName: '网页',
|
||||
sourceUrl: 'https://example.com/source',
|
||||
rawText: '网页输入',
|
||||
contentHash: 'web-hash'
|
||||
})
|
||||
).rejects.toThrow(EMBEDDING_MODEL_NOT_INSTALLED_ERROR)
|
||||
expect(
|
||||
(await runtime.documentsStore.listDocuments({ page: 1, pageSize: 20 })).total
|
||||
).toBe(0)
|
||||
|
||||
await writeCorruptEmbeddingModel(modelsPath)
|
||||
await fs.writeFile(
|
||||
path.join(modelPath, '.download-state.json'),
|
||||
JSON.stringify({ repoId: DEFAULT_EMBEDDING_MODEL_NAME, completedFiles: [] })
|
||||
)
|
||||
expect(modelService.isModelDownloaded(DEFAULT_EMBEDDING_MODEL_NAME)).toBe(false)
|
||||
await assertAllUnavailable()
|
||||
await fs.rm(path.join(modelPath, '.download-state.json'))
|
||||
expect(modelService.isModelDownloaded(DEFAULT_EMBEDDING_MODEL_NAME)).toBe(true)
|
||||
await expect(harness.invoke('kb_list_documents', {})).rejects.toThrow(
|
||||
'向量模型初始化失败'
|
||||
)
|
||||
expect(runtime.embeddingService.getStatus()).toBe('error')
|
||||
await runtime.embeddingService.close()
|
||||
await fs.rm(modelPath, { recursive: true, force: true })
|
||||
|
||||
await copyInstalledEmbeddingModel(modelsPath, sharedModelsPath)
|
||||
for (const strategy of ['auto', 'semantic', 'keyword']) {
|
||||
expect(
|
||||
await harness.invoke('kb_search', { queryText: '不存在的资料', strategy })
|
||||
).toEqual({ items: [], total: 0 })
|
||||
}
|
||||
expect(await harness.invoke('kb_list_documents', {})).toEqual({ items: [], total: 0 })
|
||||
await runtime.seedKb([
|
||||
{
|
||||
id: 'global-web',
|
||||
name: '全局网页',
|
||||
sourceType: 'web',
|
||||
sourceUrl: 'https://example.com/global',
|
||||
chunks: [{ id: 'global-chunk', text: '全局网页包含知识库可用性边界资料。' }]
|
||||
}
|
||||
])
|
||||
expect((await runtime.storageService.workspace.listWorkspaces()).length).toBe(0)
|
||||
const webResult = await harness.invoke<{
|
||||
items: Array<{ sourceUrl?: string }>
|
||||
total: number
|
||||
}>('kb_search', { queryText: '全局网页', strategy: 'auto', sourceType: 'web' })
|
||||
expect(webResult.total).toBeGreaterThan(0)
|
||||
expect(webResult.items[0].sourceUrl).toBe('https://example.com/global')
|
||||
for (const strategy of ['auto', 'semantic', 'keyword']) {
|
||||
for (const documentIds of [[], ['missing-document']]) {
|
||||
expect(
|
||||
await harness.invoke('kb_search', { queryText: '全局网页', strategy, documentIds })
|
||||
).toEqual({ items: [], total: 0 })
|
||||
}
|
||||
}
|
||||
expect(
|
||||
await harness.invoke('kb_search', {
|
||||
queryText: 'NO_MATCH_XYZ_314159',
|
||||
strategy: 'keyword'
|
||||
})
|
||||
).toEqual({ items: [], total: 0 })
|
||||
expect(
|
||||
await harness.invoke('kb_list_documents', { documentIds: ['missing-document'] })
|
||||
).toEqual({ items: [], total: 0 })
|
||||
|
||||
// A runtime cache must not bypass a fresh installation check.
|
||||
await fs.rename(modelPath, `${modelPath}-hidden`)
|
||||
await expect(
|
||||
harness.invoke('kb_search', { queryText: '全局网页', strategy: 'keyword' })
|
||||
).rejects.toThrow(EMBEDDING_MODEL_NOT_INSTALLED_ERROR)
|
||||
await fs.rename(`${modelPath}-hidden`, modelPath)
|
||||
const downloadService = VectorModelDownloadService.getInstance({
|
||||
modelService,
|
||||
embeddingService: runtime.embeddingService
|
||||
})
|
||||
expect(await downloadService.deleteModel(DEFAULT_EMBEDDING_MODEL_NAME)).toEqual({
|
||||
success: true
|
||||
})
|
||||
expect(runtime.embeddingService.getStatus()).toBe('uninitialized')
|
||||
await assertAllUnavailable()
|
||||
await copyInstalledEmbeddingModel(modelsPath, sharedModelsPath)
|
||||
expect(
|
||||
(
|
||||
await harness.invoke<{ items: Array<{ id: string }>; total: number }>(
|
||||
'kb_list_documents',
|
||||
{}
|
||||
)
|
||||
).total
|
||||
).toBe(1)
|
||||
runtime.assertThat(true, {
|
||||
name: 'kb-availability-boundaries',
|
||||
message: '模型不可用明确失败;空库正常为空;全局网页与删除恢复可用;未触发隐式下载。'
|
||||
})
|
||||
}
|
||||
})
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -1,4 +1,4 @@
|
||||
import { describe, it } from 'vitest'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { buildModelToolResultDelivery } from '../../../../src/main/services/ai/human/shared/modelToolResult'
|
||||
import {
|
||||
buildKbCapabilityTools,
|
||||
@@ -65,6 +65,33 @@ describe('knowledge foundation', () => {
|
||||
(error: unknown) => (error instanceof Error ? error.message : String(error))
|
||||
)
|
||||
|
||||
const jobStore = runtime.storageService.workspaceIndexJob
|
||||
const baseJob = await jobStore.getJobById('kb-indexing-running-prepare')
|
||||
if (!baseJob) throw new Error('missing prepare job')
|
||||
await jobStore.saveJob({ ...baseJob, source_kind: 'web' })
|
||||
for (const name of ['kb_search', 'kb_list_documents'] as const) {
|
||||
await expect(
|
||||
invokeTool(ctx, name, name === 'kb_search' ? { queryText: '合同' } : {})
|
||||
).rejects.toThrow(KB_INDEXING_IN_PROGRESS_ERROR)
|
||||
}
|
||||
await runtime.prepareEmbeddingModel()
|
||||
for (const action of ['scan', 'prepare'] as const) {
|
||||
for (const status of ['succeeded', 'failed', 'cancelled'] as const) {
|
||||
await jobStore.saveJob({ ...baseJob, action, status })
|
||||
expect(await invokeTool(ctx, 'kb_search', { queryText: '合同' })).toEqual({
|
||||
items: [],
|
||||
total: 0
|
||||
})
|
||||
expect(await invokeTool(ctx, 'kb_list_documents', {})).toEqual({ items: [], total: 0 })
|
||||
}
|
||||
}
|
||||
await jobStore.saveJob({ ...baseJob, action: 'scan', status: 'running' })
|
||||
expect(await invokeTool(ctx, 'kb_search', { queryText: '合同' })).toEqual({
|
||||
items: [],
|
||||
total: 0
|
||||
})
|
||||
expect(await invokeTool(ctx, 'kb_list_documents', {})).toEqual({ items: [], total: 0 })
|
||||
|
||||
const deliveries = []
|
||||
for (const toolName of ['kb_search', 'kb_list_documents'] as const) {
|
||||
const delivery = await buildModelToolResultDelivery({
|
||||
@@ -100,7 +127,10 @@ describe('knowledge foundation', () => {
|
||||
runtime.assertThat(
|
||||
result.deliveries.length === 2 &&
|
||||
result.deliveries.every(
|
||||
(item) => item.ok === false && item.content === MODEL_FACING_INDEXING_ERROR
|
||||
(item) =>
|
||||
item.ok === false &&
|
||||
item.content.startsWith(MODEL_FACING_INDEXING_ERROR) &&
|
||||
item.content.includes('不要继续调用 KB')
|
||||
),
|
||||
{
|
||||
name: 'indexing-error-is-model-facing',
|
||||
|
||||
@@ -17,7 +17,7 @@ describe('knowledge foundation', () => {
|
||||
fixtures: [],
|
||||
runMode: 'single',
|
||||
execute: async (runtime) => {
|
||||
await runtime.ensureEmbeddingReady()
|
||||
await runtime.prepareEmbeddingModel()
|
||||
const prepareService = new WorkspaceKnowledgePrepareService({
|
||||
embeddingService: runtime.embeddingService,
|
||||
documentsStore: runtime.documentsStore,
|
||||
@@ -73,8 +73,7 @@ describe('knowledge foundation', () => {
|
||||
)
|
||||
runtime.assertThat(result.cancelCalls.join(',') === 'ws-a', {
|
||||
name: 'prepare-stops-after-cancelled',
|
||||
message:
|
||||
'首个地址 cancelled 后不得再启动后续地址,对齐 SET-WORKSPACE-004-AC15。',
|
||||
message: '首个地址 cancelled 后不得再启动后续地址,对齐 SET-WORKSPACE-004-AC15。',
|
||||
data: result
|
||||
})
|
||||
}
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
import { describe, expect, it, vi } from 'vitest'
|
||||
import { buildKbCapabilityTools } from '../../../../src/main/services/ai/human/tools/kb/retrievalTools'
|
||||
import { runBackendCase } from '../../../harness/backend/run-backend-case'
|
||||
|
||||
describe('KB query failures', () => {
|
||||
it('never falls back to keyword search after an embedding failure', async () => {
|
||||
await runBackendCase({
|
||||
caseId: 'kb-retrieval-errors',
|
||||
domain: 'unit',
|
||||
suite: 'ai',
|
||||
harnessMode: 'tool-real',
|
||||
fixtures: [],
|
||||
runMode: 'single',
|
||||
description: '向量计算故障不能变成关键词结果。',
|
||||
assert: async () => {},
|
||||
execute: async (runtime) => {
|
||||
const ctx = runtime.createAiRunContext()
|
||||
// Unit-only fault injection isolates the query failure from availability checks.
|
||||
const available = vi.spyOn(ctx, 'assertKbAvailable').mockResolvedValue(undefined)
|
||||
const embed = vi
|
||||
.spyOn(runtime.embeddingService, 'embed')
|
||||
.mockRejectedValue(new Error('embedding computation failed'))
|
||||
const keyword = vi.spyOn(runtime.documentsStore, 'ftsSearch')
|
||||
try {
|
||||
const search = buildKbCapabilityTools({ ctx, runId: runtime.runId }).find(
|
||||
(tool) => tool.name === 'kb_search'
|
||||
)!
|
||||
for (const strategy of ['auto', 'semantic']) {
|
||||
await expect(search.invoke({ queryText: 'query', strategy })).rejects.toThrow(
|
||||
'embedding computation failed'
|
||||
)
|
||||
}
|
||||
expect(keyword).not.toHaveBeenCalled()
|
||||
} finally {
|
||||
available.mockRestore()
|
||||
embed.mockRestore()
|
||||
keyword.mockRestore()
|
||||
}
|
||||
}
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -1045,29 +1045,19 @@ describe('unit ai persistence backend', () => {
|
||||
)
|
||||
|
||||
it('autostart-run-emits-process-events-after-run-started', async () => {
|
||||
const previousTimeout = process.env.NCURATOR_AI_EMBEDDING_PREFLIGHT_TIMEOUT_MS
|
||||
process.env.NCURATOR_AI_EMBEDDING_PREFLIGHT_TIMEOUT_MS = '10'
|
||||
try {
|
||||
const setup = await createTestService({ autoStartRuns: true })
|
||||
const run = await submitAiRun(setup)
|
||||
const setup = await createTestService({ autoStartRuns: true })
|
||||
const run = await submitAiRun(setup)
|
||||
|
||||
await waitForCondition(() =>
|
||||
setup.events.some((event) => event.type === 'assistant_process_group_started')
|
||||
)
|
||||
await waitForCondition(() => setup.events.some((event) => event.type === 'run_failed'))
|
||||
await waitForCondition(() =>
|
||||
setup.events.some((event) => event.type === 'assistant_process_group_started')
|
||||
)
|
||||
await waitForCondition(() => setup.events.some((event) => event.type === 'run_failed'))
|
||||
|
||||
const storedRun = await setup.storage.aiRun.getRun(run.id)
|
||||
const storedRun = await setup.storage.aiRun.getRun(run.id)
|
||||
|
||||
expect(setup.events.map((event) => event.type)).toContain('run_started')
|
||||
expect(setup.events.map((event) => event.type)).toContain('assistant_process_group_started')
|
||||
expect(storedRun?.status).toBe('failed')
|
||||
} finally {
|
||||
if (previousTimeout === undefined) {
|
||||
delete process.env.NCURATOR_AI_EMBEDDING_PREFLIGHT_TIMEOUT_MS
|
||||
} else {
|
||||
process.env.NCURATOR_AI_EMBEDDING_PREFLIGHT_TIMEOUT_MS = previousTimeout
|
||||
}
|
||||
}
|
||||
expect(setup.events.map((event) => event.type)).toContain('run_started')
|
||||
expect(setup.events.map((event) => event.type)).toContain('assistant_process_group_started')
|
||||
expect(storedRun?.status).toBe('failed')
|
||||
})
|
||||
|
||||
it('does-not-attribute-uncontrolled-workspace-changes-when-a-run-finishes', async () => {
|
||||
|
||||
@@ -1,5 +1,8 @@
|
||||
import { describe, expect, it, vi } from 'vitest'
|
||||
import { EmbeddingService, type EmbeddingExecutor } from '../../../../../src/main/services/vector/EmbeddingService'
|
||||
import {
|
||||
EmbeddingService,
|
||||
type EmbeddingExecutor
|
||||
} from '../../../../../src/main/services/vector/EmbeddingService'
|
||||
|
||||
const result = {
|
||||
data: new Float32Array([1, 2]),
|
||||
@@ -79,3 +82,24 @@ describe('EmbeddingService', () => {
|
||||
expect(initialize).toHaveBeenCalledTimes(2)
|
||||
})
|
||||
})
|
||||
|
||||
it('waits for initialization before closing and clears the ready cache', async () => {
|
||||
let finish!: () => void
|
||||
const executor: EmbeddingExecutor = {
|
||||
initialize: () =>
|
||||
new Promise<void>((resolve) => {
|
||||
finish = resolve
|
||||
}),
|
||||
embed: async () => result,
|
||||
close: vi.fn(async () => undefined)
|
||||
}
|
||||
const service = new EmbeddingService({ executor })
|
||||
const loading = service.initialize()
|
||||
const closing = service.close()
|
||||
expect(executor.close).not.toHaveBeenCalled()
|
||||
finish()
|
||||
await loading
|
||||
await closing
|
||||
expect(executor.close).toHaveBeenCalledTimes(1)
|
||||
expect(service.getStatus()).toBe('uninitialized')
|
||||
})
|
||||
@@ -101,7 +101,7 @@ describe('unit vector download service contract', () => {
|
||||
return { success: true }
|
||||
}
|
||||
)
|
||||
vi.spyOn(embeddingService, 'initialize').mockResolvedValue()
|
||||
const initializeSpy = vi.spyOn(embeddingService, 'initialize')
|
||||
|
||||
const unsubscribe = service.onProgress((payload) => {
|
||||
progressEvents.push(payload)
|
||||
@@ -112,6 +112,8 @@ describe('unit vector download service contract', () => {
|
||||
releaseFirstDownload?.()
|
||||
const firstResult = await firstTask
|
||||
unsubscribe()
|
||||
if (initializeSpy.mock.calls.length !== 0)
|
||||
throw new Error('download must not initialize embedding')
|
||||
|
||||
return {
|
||||
firstResult,
|
||||
@@ -234,7 +236,7 @@ describe('unit vector download service contract', () => {
|
||||
}
|
||||
)
|
||||
const deleteSpy = vi.spyOn(modelService, 'deleteModel').mockResolvedValue({ success: true })
|
||||
vi.spyOn(embeddingService, 'initialize').mockResolvedValue()
|
||||
const initializeSpy = vi.spyOn(embeddingService, 'initialize')
|
||||
|
||||
const unsubscribe = service.onProgress((payload) => {
|
||||
progressEvents.push(payload)
|
||||
@@ -246,6 +248,8 @@ describe('unit vector download service contract', () => {
|
||||
const deleteResult = await service.deleteModel(REPO_A)
|
||||
const downloadResult = await downloadTask
|
||||
unsubscribe()
|
||||
if (initializeSpy.mock.calls.length !== 0)
|
||||
throw new Error('download must not initialize embedding')
|
||||
|
||||
return {
|
||||
downloadResult,
|
||||
@@ -375,7 +379,7 @@ describe('unit vector download service contract', () => {
|
||||
return { success: true }
|
||||
})
|
||||
vi.spyOn(modelService, 'isModelDownloaded').mockReturnValue(false)
|
||||
vi.spyOn(embeddingService, 'initialize').mockResolvedValue()
|
||||
const initializeSpy = vi.spyOn(embeddingService, 'initialize')
|
||||
|
||||
const unsubscribe = service.onProgress((payload) => {
|
||||
progressEvents.push(payload)
|
||||
@@ -395,6 +399,8 @@ describe('unit vector download service contract', () => {
|
||||
}
|
||||
await waitForCondition(() => service.getActiveDownload() === null)
|
||||
unsubscribe()
|
||||
if (initializeSpy.mock.calls.length !== 0)
|
||||
throw new Error('download must not initialize embedding')
|
||||
|
||||
return {
|
||||
resumedRepoId: downloadSpy.mock.calls[0]?.[0] || '',
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
import fs from 'node:fs/promises'
|
||||
import os from 'node:os'
|
||||
import path from 'node:path'
|
||||
import { describe, it } from 'vitest'
|
||||
import { runE2ECase } from '../../../../harness/e2e/run-e2e-case'
|
||||
import {
|
||||
WORKBENCH_LIVE_AI_E2E_TIMEOUT_MS,
|
||||
configureLiveModelThroughSettingsUi,
|
||||
createPartnerThroughWorkbenchUi,
|
||||
navigateToWorkbench,
|
||||
selectWorkbenchModel,
|
||||
sendWorkbenchMessage,
|
||||
waitForWorkbenchAiRunCompletion,
|
||||
loadWorkbenchLiveModelEnv
|
||||
} from '../../../../harness/e2e/workbench'
|
||||
|
||||
const CASE_ID = 'chat-without-knowledge-model'
|
||||
|
||||
describe('workbench conversation', () => {
|
||||
it(
|
||||
CASE_ID,
|
||||
async () => {
|
||||
const modelsPath = await fs.mkdtemp(path.join(os.tmpdir(), 'ncurator-chat-empty-models-'))
|
||||
try {
|
||||
await runE2ECase({
|
||||
caseId: CASE_ID,
|
||||
domain: 'workbench',
|
||||
suite: 'conversation',
|
||||
groups: ['composer'],
|
||||
description: '无工作空间、无向量模型仍可完成真实聊天,且不下载或启动 embedding。',
|
||||
fixtures: [],
|
||||
serviceMode: 'real',
|
||||
execution: 'live',
|
||||
windowMode: 'background',
|
||||
runMode: 'single',
|
||||
extraEnv: { NCURATOR_MODELS_PATH: modelsPath },
|
||||
execute: async (runtime) => {
|
||||
const { page } = runtime
|
||||
const target = loadWorkbenchLiveModelEnv()
|
||||
await configureLiveModelThroughSettingsUi(page, target)
|
||||
await navigateToWorkbench(page)
|
||||
await createPartnerThroughWorkbenchUi(page, '普通聊天')
|
||||
await selectWorkbenchModel(page, target.modelName)
|
||||
const startedAt = Date.now()
|
||||
await sendWorkbenchMessage(page, '你好,请用一句话回复我。')
|
||||
const evidence = await waitForWorkbenchAiRunCompletion(
|
||||
runtime,
|
||||
startedAt,
|
||||
target.modelName
|
||||
)
|
||||
const state = await page.evaluate(async () => ({
|
||||
embedding: await window.api.getEmbeddingStatus(),
|
||||
download: await window.api.getActiveDownload()
|
||||
}))
|
||||
return {
|
||||
state,
|
||||
files: await fs.readdir(modelsPath, { recursive: true }),
|
||||
completed: evidence.events.some((event) => event.type === 'run_completed')
|
||||
}
|
||||
},
|
||||
assert: async (result, runtime) => {
|
||||
await runtime.assertThat(
|
||||
result.completed &&
|
||||
result.files.length === 0 &&
|
||||
result.state.download === null &&
|
||||
result.state.embedding.status === 'uninitialized' &&
|
||||
result.state.embedding.pid === null,
|
||||
{
|
||||
name: 'chat-without-knowledge-model',
|
||||
message:
|
||||
'真实聊天完成,空模型目录未变化、无下载任务或 embedding 进程,对齐 WB-COMPOSER-001-AC5。',
|
||||
data: result
|
||||
}
|
||||
)
|
||||
}
|
||||
})
|
||||
} finally {
|
||||
await fs.rm(modelsPath, { recursive: true, force: true })
|
||||
}
|
||||
},
|
||||
WORKBENCH_LIVE_AI_E2E_TIMEOUT_MS
|
||||
)
|
||||
})
|
||||
@@ -5,6 +5,7 @@ import { AiRunService } from '../../../src/main/services/ai/AiRunService'
|
||||
import { AiFileChangeService } from '../../../src/main/services/ai/fileChanges/AiFileChangeService'
|
||||
import { createScriptedStructuredOutputDriver } from './scriptedStructuredOutputDriver'
|
||||
import type { BackendTestRuntime, CreateAiRunContextInput } from './test-types'
|
||||
import { ensureEmbeddingReady } from '../../../src/main/services/vector/ensureEmbeddingReady'
|
||||
import type { AiRunStartPayload, AiRunDeps } from '../../../src/main/services/ai/types'
|
||||
|
||||
/** Uses the same top-level runtime as ChatRunService, including subagent dispatch and cleanup. */
|
||||
@@ -39,7 +40,8 @@ export const createAiServiceTestRunner = async (
|
||||
chatMessageLogStore: runtime.storageService.chatMessageLog,
|
||||
documentsStore: runtime.documentsStore,
|
||||
embeddingService: runtime.embeddingService,
|
||||
ensureEmbeddingReady: runtime.ensureEmbeddingReady,
|
||||
ensureEmbeddingReady: () =>
|
||||
ensureEmbeddingReady({ embeddingService: runtime.embeddingService }),
|
||||
docxApplyService: runtime.docxApplyService,
|
||||
saveFileChangeRecord: (record) => files.saveRecord(record),
|
||||
listFileChangeRecords: async (request) => (await files.listRecords(request)).items,
|
||||
@@ -91,7 +93,8 @@ export const createChatServiceTestRunner = async (
|
||||
workspaceStore: runtime.storageService.workspace,
|
||||
workspaceIndexJobStore: runtime.storageService.workspaceIndexJob,
|
||||
webSourceStore: runtime.storageService.webSource,
|
||||
ensureEmbeddingReady: runtime.ensureEmbeddingReady,
|
||||
ensureEmbeddingReady: () =>
|
||||
ensureEmbeddingReady({ embeddingService: runtime.embeddingService }),
|
||||
docxApplyService: runtime.docxApplyService,
|
||||
aiFileChangeService: files,
|
||||
runResourceManager: new AiRunResourceManager(),
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import { createHumanGraphRunner } from '../../../src/main/services/ai/human/graph/humanGraph'
|
||||
import { runAiRuntimePreflight } from '../../../src/main/services/ai/runtimePreflight'
|
||||
import { AiRunResourceManager } from '../../../src/main/services/ai/run/AiRunResourceManager'
|
||||
import { releaseDocxRunSession } from '../../../src/main/services/ai/human/tools/docx/worker/docxWorkerClient'
|
||||
import { releasePdfRunSession } from '../../../src/main/services/pdf/worker/pdfWorkerClient'
|
||||
@@ -27,9 +26,6 @@ export const createHumanTestRunner = (
|
||||
ctx,
|
||||
runner: run,
|
||||
run: async (taskInput: HumanTaskInput, dispatchOptions?: HumanRunDispatchOptions) => {
|
||||
await runAiRuntimePreflight({
|
||||
ensureEmbeddingReady: runtime.ensureEmbeddingReady
|
||||
})
|
||||
try {
|
||||
return (await run(taskInput, dispatchOptions)) as HumanTaskOutput
|
||||
} finally {
|
||||
|
||||
@@ -0,0 +1,66 @@
|
||||
import fs from 'node:fs/promises'
|
||||
import { constants } from 'node:fs'
|
||||
import os from 'node:os'
|
||||
import path from 'node:path'
|
||||
import { getModelsPath } from '../../../src/main/utils/paths'
|
||||
import { ModelService } from '../../../src/main/services/model/ModelService'
|
||||
import { DEFAULT_EMBEDDING_MODEL_NAME } from '../../../src/main/services/vector/EmbeddingService'
|
||||
|
||||
/** Isolates real model files without replacing production availability checks. */
|
||||
export const withIsolatedModels = async (
|
||||
run: (modelsPath: string, sharedModelsPath: string) => Promise<void>
|
||||
): Promise<void> => {
|
||||
const previous = process.env.NCURATOR_MODELS_PATH
|
||||
const sharedModelsPath = getModelsPath()
|
||||
const modelsPath = await fs.mkdtemp(path.join(os.tmpdir(), 'ncurator-model-boundary-'))
|
||||
process.env.NCURATOR_MODELS_PATH = modelsPath
|
||||
try {
|
||||
await run(modelsPath, sharedModelsPath)
|
||||
} finally {
|
||||
if (previous === undefined) delete process.env.NCURATOR_MODELS_PATH
|
||||
else process.env.NCURATOR_MODELS_PATH = previous
|
||||
await fs.rm(modelsPath, { recursive: true, force: true })
|
||||
}
|
||||
}
|
||||
|
||||
/** Explicit test setup; KB calls never download a missing model. */
|
||||
export const copyInstalledEmbeddingModel = async (
|
||||
modelsPath: string,
|
||||
sharedModelsPath: string
|
||||
): Promise<void> => {
|
||||
process.env.NCURATOR_MODELS_PATH = sharedModelsPath
|
||||
try {
|
||||
const service = ModelService.getInstance()
|
||||
if (!service.isModelDownloaded(DEFAULT_EMBEDDING_MODEL_NAME)) {
|
||||
await service.downloadModel(DEFAULT_EMBEDDING_MODEL_NAME)
|
||||
}
|
||||
await fs.cp(
|
||||
path.join(sharedModelsPath, DEFAULT_EMBEDDING_MODEL_NAME),
|
||||
path.join(modelsPath, DEFAULT_EMBEDDING_MODEL_NAME),
|
||||
{ recursive: true, mode: constants.COPYFILE_FICLONE }
|
||||
)
|
||||
} finally {
|
||||
process.env.NCURATOR_MODELS_PATH = modelsPath
|
||||
}
|
||||
}
|
||||
|
||||
/** Non-empty required files pass installation checks but fail real Transformers initialization. */
|
||||
export const writeCorruptEmbeddingModel = async (modelsPath: string): Promise<void> => {
|
||||
for (const name of [
|
||||
'1_Pooling/config.json',
|
||||
'config.json',
|
||||
'config_sentence_transformers.json',
|
||||
'merges.txt',
|
||||
'modules.json',
|
||||
'onnx/model.onnx',
|
||||
'sentence_bert_config.json',
|
||||
'special_tokens_map.json',
|
||||
'tokenizer.json',
|
||||
'tokenizer_config.json',
|
||||
'vocab.json'
|
||||
]) {
|
||||
const target = path.join(modelsPath, DEFAULT_EMBEDDING_MODEL_NAME, name)
|
||||
await fs.mkdir(path.dirname(target), { recursive: true })
|
||||
await fs.writeFile(target, 'invalid model file')
|
||||
}
|
||||
}
|
||||
@@ -5,7 +5,6 @@ import { AiFileChangeService } from '../../../src/main/services/ai/fileChanges/A
|
||||
import { DocxApplyService } from '../../../src/main/services/ai/human/tools/docx/review/DocxApplyService'
|
||||
import { terminalSessionManager } from '../../../src/main/services/ai/human/tools/terminal/TerminalSessionManager'
|
||||
import { createAiRunContext } from '../../../src/main/services/ai/runtime'
|
||||
import { runAiRuntimePreflight } from '../../../src/main/services/ai/runtimePreflight'
|
||||
import type {
|
||||
AiApprovalBatchRequest,
|
||||
AiApprovalBatchResult
|
||||
@@ -107,6 +106,7 @@ export type RealAiDepsFactory = {
|
||||
workspaceKnowledgePrepareService: WorkspaceKnowledgePrepareService
|
||||
docxApplyService: DocxApplyService
|
||||
ensureEmbeddingReady: () => Promise<void>
|
||||
prepareEmbeddingModel: () => Promise<void>
|
||||
createAiRunContext: (input?: CreateAiRunContextInput) => AiRunContext
|
||||
createHumanTaskRunner: (
|
||||
input?: CreateAiRunContextInput
|
||||
@@ -324,14 +324,18 @@ export const createRealAiDepsFactory = async (input: {
|
||||
workspaceKnowledgePrepareService,
|
||||
docxApplyService,
|
||||
ensureEmbeddingReady: ensureReady,
|
||||
prepareEmbeddingModel: async () => {
|
||||
const modelName = embeddingService.getModelName()
|
||||
if (!modelService.isModelDownloaded(modelName)) {
|
||||
await modelService.downloadModel(modelName)
|
||||
}
|
||||
await ensureReady()
|
||||
},
|
||||
createAiRunContext: createContext,
|
||||
createHumanTaskRunner: (contextInput = {}) => {
|
||||
const ctx = createContext(contextInput)
|
||||
const run = createHumanGraphRunner({ ctx })
|
||||
return async (taskInput, options) => {
|
||||
await runAiRuntimePreflight({
|
||||
ensureEmbeddingReady: ensureReady
|
||||
})
|
||||
return await run(taskInput, options)
|
||||
}
|
||||
},
|
||||
|
||||
@@ -87,8 +87,9 @@ export const createRealAiHarness = async (input: {
|
||||
ensureEmbeddingReady: async () => {
|
||||
await depsFactory.ensureEmbeddingReady()
|
||||
},
|
||||
prepareEmbeddingModel: depsFactory.prepareEmbeddingModel,
|
||||
seedKb: async (documents) => {
|
||||
await depsFactory.ensureEmbeddingReady()
|
||||
await depsFactory.prepareEmbeddingModel()
|
||||
await seedKbFixture({
|
||||
documentsStore: depsFactory.storageService.documents,
|
||||
embeddingService: depsFactory.embeddingService,
|
||||
@@ -97,7 +98,7 @@ export const createRealAiHarness = async (input: {
|
||||
appendSeedKbDocuments(seedManifest, documents)
|
||||
},
|
||||
seedWorkspaceKb: async (params) => {
|
||||
await depsFactory.ensureEmbeddingReady()
|
||||
await depsFactory.prepareEmbeddingModel()
|
||||
const seeded = await seedWorkspaceKb({
|
||||
runtime,
|
||||
params
|
||||
|
||||
@@ -239,6 +239,7 @@ export type BackendTestRuntime = {
|
||||
createAiRunContext: (input?: CreateAiRunContextInput) => AiRunContext
|
||||
createHumanTaskRunner: (input?: CreateAiRunContextInput) => HumanTaskRunner
|
||||
ensureEmbeddingReady: () => Promise<void>
|
||||
prepareEmbeddingModel: () => Promise<void>
|
||||
seedKb: (input: readonly SeedKbDocumentInput[]) => Promise<void>
|
||||
seedWorkspaceKb: (input: WorkspaceKbSeedInput) => Promise<WorkspaceKbSeedResult[]>
|
||||
seedChatHistory: (input: SeedChatHistoryInput) => Promise<void>
|
||||
|
||||
Reference in new issue
Block a user