Refactor macOS release handling: remove x64 support and streamline metadata processing

- Removed support for macos-x64 in fragment definitions and related scripts.
- Updated finalize-macos script to only handle macos-arm64, simplifying signature checks and metadata writing.
- Deleted merge-mac-metadata script as it is no longer needed.
- Adjusted storage layout and release status checks to reflect the removal of x64.
- Updated tests to ensure they only validate macos-arm64 functionality.
This commit is contained in:
hy committed 2026-09-01 16:36:09 +08:00
1 parent c193edf338
commit 7cd3782f1e
25 files changed
+288 -452

No files matched your search

-6
View File
@@ -8,12 +8,6 @@
"appAsar": { "baselineMiB": 230.9, "hardLimitMiB": 252 },
"dmg": { "baselineMiB": 432.9, "hardLimitMiB": 480 },
"zip": { "baselineMiB": 418.5, "hardLimitMiB": 465 }
},
"darwin-x64": {
"installedApp": { "baselineMiB": 1145.6, "hardLimitMiB": 1230 },
"appAsar": { "baselineMiB": 230.9, "hardLimitMiB": 252 },
"dmg": { "baselineMiB": 447.9, "hardLimitMiB": 495 },
"zip": { "baselineMiB": 433.3, "hardLimitMiB": 480 }
}
}
}
+5 -18
View File
@@ -1,6 +1,6 @@
# 桌面端构建与发布
本文是 Windows、macOS 构建、正式包回归和对象存储发布的统一说明。国内默认发布到腾讯云 COS,海外可显式切换到 Cloudflare R2。当前发布 Windows x64、macOS arm64 和 macOS x64,不发布 Linux。Windows 使用 `windows-vX.Y.Z`,macOS 使用 `mac-vX.Y.Z`,两个平台独立递增和发布。
本文是 Windows、macOS 构建、正式包回归和对象存储发布的统一说明。国内默认发布到腾讯云 COS,海外可显式切换到 Cloudflare R2。当前只发布 Windows x64 和 macOS arm64,不支持 Intel Mac,也不发布 Linux。Windows 使用 `windows-vX.Y.Z`,macOS 使用 `mac-vX.Y.Z`,两个平台独立递增和发布。
注意评估改动影响,只回归需要回归的测试用例,避免动不动就全部回归,尤其是在回归一次后,修复的代码并不影响已通过的测试用例时。
@@ -92,13 +92,9 @@ pnpm release:mac -- --tag mac-vX.Y.Z --test-only --case <caseId-1> --case <caseI
回归范围由执行者根据改动影响评估决定:影响面广时跑全量,修复后的定向复测用 `--case` 列出受影响的用例。`--case` 指定的用例必须能匹配到测试文件,匹配不到时 runner 直接失败,不会静默通过。
macOS arm64 候选包必须通过选定的回归范围。macOS x64 仍正常构建、签名、审计、启动验证和发布,但由于 Rosetta 下正式包 E2E 速度过慢,默认不执行 x64 正式包 E2E,也不把它作为发布门禁。需要人工诊断时可显式执行 `--include-x64-e2e`,x64 E2E 固定单并发:
macOS arm64 候选包必须通过选定的回归范围。项目不再构建、测试或发布 Intel Mac 安装包。
```bash
pnpm release:mac -- --tag mac-vX.Y.Z --test-only --include-x64-e2e
```
所有发布阶段最多运行 60 分钟;测试 runner 在 55 分钟停止领取新用例并保存部分结果。正式并发开始前,每个实际参加 E2E 的候选包先串行通过唯一的 `warmup` 向量索引用例,让模型和磁盘缓存完成冷启动;warm-up 产生的 split/vector 缓存会作为种子复制到后续 case 的隔离缓存目录,warm-up 失败时不会启动后续并发。使用 `--domain`/`--suite`/`--case` 收窄回归范围时,只要本次执行的 case 数超过 1,runner 也会自动纳入全仓唯一的 `warmup` 用例并先预热,不会因过滤漏掉它。随后 E2E 总并发和 live 并发上限均为 3,原生对话框、鼠标和桌面焦点用例独占执行。显式启用 macOS x64 E2E 时,x64 仍固定为单并发。
所有发布阶段最多运行 60 分钟;测试 runner 在 55 分钟停止领取新用例并保存部分结果。正式并发开始前,每个实际参加 E2E 的候选包先串行通过唯一的 `warmup` 向量索引用例,让模型和磁盘缓存完成冷启动;warm-up 产生的 split/vector 缓存会作为种子复制到后续 case 的隔离缓存目录,warm-up 失败时不会启动后续并发。使用 `--domain`/`--suite`/`--case` 收窄回归范围时,只要本次执行的 case 数超过 1,runner 也会自动纳入全仓唯一的 `warmup` 用例并先预热,不会因过滤漏掉它。随后 E2E 总并发和 live 并发上限均为 3,原生对话框、鼠标和桌面焦点用例独占执行。
选中的正式包 E2E 全部执行,其中使用真实 AI 或真实服务的 E2E 也不排除。源码 E2E、backend、unit 和独立的 `test:ai*` 测试集只用于开发验证或问题排查,不替代正式包 E2E,也不属于标准发布门。
@@ -146,12 +142,7 @@ Windows 本地发布环境:
### macOS
macOS 没有 Actions 构建路线。prepare 必须在 Apple Silicon Mac 执行,并从同一 Tag 分别导出 arm64 与 x64 两份干净代码。
- arm64 使用本机 arm64 Node/Python。
- x64 使用 Rosetta、x64 Node 和 x64 Python。
- 两个架构分别安装依赖、构建、签名、审计和生成候选包。
- arm64 候选包必须完成正式包 E2E;x64 默认跳过正式包 E2E,但发布时仍会检查两个架构的更新签名一致。
macOS 没有 Actions 构建路线。prepare 必须在 Apple Silicon Mac 执行,使用本机 arm64 Node/Python 安装依赖、构建、签名、审计和生成唯一的 arm64 候选包。该候选包必须完成正式包 E2E。
```bash
git fetch --tags
@@ -186,7 +177,7 @@ release-stage/
└── resources/{native,web_crawler_sidecar,open-with}/
```
安装包不得包含源码、测试、文档、开发配置、`.env`、锁文件、Source Map、测试日志、fixture、私钥或错误平台文件。网页抓取 sidecar 不得包含 `.local-browsers` 或 `chromium_headless_shell-*`,且解包体积不得超过 340 MiB;staging 和正式包审计都会硬校验。`app.asar` 不是加密,所有 `VITE_*` 变量都按公开信息处理。
安装包不得包含源码、测试、文档、开发配置、`.env`、锁文件、Source Map、测试日志、fixture、私钥或错误平台文件。网页抓取 sidecar 不得包含 `.local-browsers` 或 `chromium_headless_shell-*`,且解包体积不得超过 400 MiB;staging 和正式包审计都会硬校验。`app.asar` 不是加密,所有 `VITE_*` 变量都按公开信息处理。
候选 bundle 绑定 Tag、commit、release fragment、安装包、更新文件、大小、sha256 和 sha512。测试和发布前都会重新验证这些内容。
@@ -201,14 +192,12 @@ artifacts/windows/windows-vX.Y.Z/*
artifacts/macos/mac-vX.Y.Z/*
release-status/windows/windows-vX.Y.Z.json
release-status/macos/mac-vX.Y.Z/macos-arm64.json
release-status/macos/mac-vX.Y.Z/macos-x64.json
updates/latest/latest.yml
updates/latest/latest-mac.yml
releases/latest/windows.json
releases/latest/macos.json
downloads/latest/windows-x64.exe
downloads/latest/macos-arm64.dmg
downloads/latest/macos-x64.dmg
```
版本化产物使用不可变缓存;latest、manifest 和固定下载使用 `no-store`。fragment 中的 `name` 记录用户可见文件名,`storageName` 记录对象存储的 ASCII 对象名;二者共同进入候选 bundle 校验。Windows 固定下载与 manifest 返回 `馆长-X.Y.Z-Windows-x64.exe`,自动更新 metadata 则指向对应的 ASCII 对象键。
@@ -260,7 +249,6 @@ Windows 清单:https://download.guanzhangai.cn/releases/latest/windows.json
macOS 清单:https://download.guanzhangai.cn/releases/latest/macos.json
Windows 下载:https://download.guanzhangai.cn/downloads/latest/windows-x64.exe
macOS arm64:https://download.guanzhangai.cn/downloads/latest/macos-arm64.dmg
macOS x64:https://download.guanzhangai.cn/downloads/latest/macos-x64.dmg
Windows 更新:https://download.guanzhangai.cn/updates/latest/latest.yml
macOS 更新:https://download.guanzhangai.cn/updates/latest/latest-mac.yml
```
@@ -295,7 +283,6 @@ pnpm run build
pnpm run build:unpack
pnpm run build:win
pnpm run build:mac:arm64
pnpm run build:mac:x64
pnpm run package:audit
pnpm run package:audit:summary
```
+13 -4
View File
@@ -22,8 +22,8 @@
| human-legacy-doc-reading-live | 完成 | 94 | deepseek-v4-flash | 2026-08-13 19:56 CST | tests/logs/ai/human/live/human-legacy-doc-reading-live/20260813-195642-815 | 通过 docx 读取工具成功读取旧版 .doc 并准确回答五级风险与区别率,源文件未改。 |
| human-text-file-write-checkout-cutover-live | 完成 | 96 | deepseek-v4-flash | 2026-08-13 20:17 CST | tests/logs/ai/human/live/human-text-file-write-checkout-cutover-live/20260813-201727-173 | 两处端点和超时精确更新,重试与 telemetry 不变,写后核对完整。 |
| human-text-file-read-checkout-readiness-live | 完成 | 93 | deepseek-v4-flash | 2026-08-13 20:16 CST | tests/logs/ai/human/live/human-text-file-read-checkout-readiness-live/20260813-201642-554 | 只读审查准确识别就绪性风险与保留配置,源文件未改。 |
| human-web-search-by-url-live | 完成 | 95 | qwen3.8-flash | 2026-08-31 18:29 CST | tests/logs/ai/human/live/human-web-search-by-url-live/20260831-182946-081 | 指定地址直接 type=fetch 读取并回答申请对象/截止日期;日期断言已容忍空格。 |
| human-web-search-intent-query-live | 完成 | 90 | qwen3.8-flash | 2026-08-31 18:31 CST | tests/logs/ai/human/live/human-web-search-intent-query-live/20260831-183108-765 | 语句优化后 2×type=search + 2×fetch 官方 API 页,基于正文回答。 |
| human-web-search-by-url-live | 完成 | 95 | qwen3.8-flash | 2026-09-01 16:24 CST | tests/logs/ai/human/live/human-web-search-by-url-live/20260901-162353-087 | 指定地址后连续 2×fetch 通知页与指南页,事实回答正确,errors=0。 |
| human-web-search-intent-query-live | 完成 | 90 | qwen3.8-flash | 2026-09-01 16:24 CST | tests/logs/ai/human/live/human-web-search-intent-query-live/20260901-162353-087 | 2×search + 2×fetch 成功读取 read_csv 与 DataFrame.to_csv 官方 API 页,errors=0。 |
| human-terminal-preview-service-live | 完成 | 88 | qwen3.8-flash | 2026-08-31 18:30 CST | tests/logs/ai/human/live/human-terminal-preview-service-live/20260831-183019-685 | 启动预览服务、terminal_read 读请求日志、交付页面标记并关闭服务(已移除不存在的 terminal_write 断言)。 |
| human-legacy-doc-edit-skip-live | 完成 | 88 | deepseek-v4-flash | 2026-08-21 23:11 CST | tests/logs/ai/human/live/human-legacy-doc-edit-skip-live/20260821-231108-129 | .doc 写入失败后停止重试,源文件不变,任务完成。 |
| human-docx-visual-reading-live | 完成 | 92 | deepseek-v4-flash / qwen3.6-plus | 2026-08-21 23:04 CST | tests/logs/ai/human/live/human-docx-visual-reading-live/20260821-230430-243 | 真实读取 DOCX 图片并准确回答,源文件未改。 |
@@ -526,6 +526,15 @@ DOCX tool 回归以 7 个公开工具、story 内容树、稳定 `nodeId`、segm
- Oracle:结果返回明确错误文本,同时保留 `finalUrl` 和 `statusCode`。
- Not:不验证复杂重定向链或外站错误页样式。
##### `web-search-fetch-reuses-one-sidecar-for-multiple-pages`
- 测试类型:`backend`
- What:同一个冻结 sidecar、Crawl4AI 实例和 Electron CDP context 可以连续抓取多个页面。
- Why:真实 human 会在一次任务中读取多个搜索结果;只验证第一次 fetch 会漏掉连接复用和结果序列化回归。
- How:启动本地 HTTP server 和测试 Electron CDP host,复用同一个 `webCrawlerService` 依次 fetch 通知页与指南页。
- Oracle:两次调用都成功,分别返回对应 `finalUrl` 和正文,不出现重复抓取序列化错误。
- Not:不验证并发抓取或跨 run 复用。
##### `web-search-fetch-returns-full-content-without-tool-level-truncation`
- 测试类型:`backend`
@@ -976,7 +985,7 @@ DOCX tool 回归以 7 个公开工具、story 内容树、稳定 `nodeId`、segm
- What:live 模式下 `human` 会基于真实 brain usage 与新增 history 增量在风险周报长任务中自然触发 `context_compaction`,并在压缩后继续完成交付。
- Why:相比单独直调节点,live 回归更应该守住“真实 graph 中自然压缩且不影响任务完成”的能力。
- How:复用风险周报 live 的真实 workspace 与业务输入,通过 `humanRuntimeOverrides.modelContextWindowTokens=40_000` 提供可工作的上下文空间,并让真实任务在多轮资料读取后自然进入 `context_budget -> context_compaction -> context_budget -> brain`。窗口不能太小:固定开销(23 工具 schema ~18k + system/模板 ~2k ≈ 20k)几乎不随压缩减少,若窗口 24k 则触发线 19.2k 被固定开销直接顶破,压缩后仍超阈值、每轮压缩死循环;也不能太大(64k 时任务峰值 ~40k 达不到 51.2k 触发线,0 次压缩)。
- Oracle:至少发生一次完整 compaction;不限制压缩次数。逐次检查固定摘要结构、任务锚点、已确认路径、无编造和无“已完成又未读取”的进度矛盾,并检查全部压缩比;后续 `brain` prompt 保留 compaction 与 task progress;最终周报文件成功 apply。按 10 分制评估压缩生命周期、摘要结构、摘要质量、连续性和文件交付,严格超过 8 分通过。
- Oracle:至少发生一次完整 compaction;不限制压缩次数。逐次检查固定摘要结构、任务锚点、已确认路径、无编造和无“已完成又未读取”的进度矛盾,并检查全部压缩比;后续 `brain` prompt 保留 compaction 与 task progress;最终周报文件成功交付并产出 docx `applyId` 证据,且至少一次成功的 docx 写入(允许 `document_end` 新建,也允许复制底本后 `replace_text` / `insert_nodes` 改写)。按 10 分制评估压缩生命周期、摘要结构、摘要质量、连续性和文件交付,严格超过 8 分通过。
- Not:不绑定压缩次数,不要求固定资料读取顺序,也不因合理表达差异判失败。
##### `human-risk-report-draft-live`
@@ -1048,7 +1057,7 @@ DOCX tool 回归以 7 个公开工具、story 内容树、稳定 `nodeId`、segm
- What:live 模式下用户只给纯文本诉求、不给网址,`human` 提炼搜索意图后通过 `web_search(type=search)` 获取候选,并用 `type=fetch` 读取官方文档回答。
- Why:真实用户常只描述诉求不给网址;`human` 应提炼关键词走 search 而非直接 fallback 浏览器。
- How:不向用户输入提供 URL,用户以「记不清官方网址、帮忙搜索官方文档中 CSV 读取与写入 API 用法」的自然语言诉求要求搜索并阅读官方页面;要求模型自行提炼意图并读取官方文档页面完成整理。
- Oracle:至少一次成功调用 `web_search(type=search)`;至少一次成功 `web_search(type=fetch)` 读取 `pandas.pydata.org` 官方 API 页面并基于正文回答。
- Oracle:至少一次成功调用 `web_search(type=search)`;`pandas.read_csv` 与 `pandas.DataFrame.to_csv` 两个 `pandas.pydata.org` 官方 API 页面都必须成功 fetch,并基于正文回答。
- Not:不要求复杂跨域或深链路,也不要求模型第一次就命中最终文档页;因 query 搜索偶发不可用,允许模型在搜索失败时用 `browser_use` 兜底读取官方页面。
##### `human-text-file-read-checkout-readiness-live`
+1 -3
View File
@@ -51,10 +51,8 @@
"package:audit:summary": "node scripts/build/package/audit-summary.mjs dist/package-audit.json",
"build:unpack": "pnpm run build && electron-builder --dir && pnpm run package:audit",
"build:win": "node scripts/build/native/electron-builder-windows.mjs && pnpm run build && electron-builder --win --x64 --publish never && pnpm run package:audit",
"build:mac": "pnpm run build && electron-builder --mac --publish never && pnpm run package:audit",
"build:mac:x64": "pnpm run build && electron-builder --mac --x64 --publish never && pnpm run package:audit",
"build:mac": "pnpm run build && electron-builder --mac --arm64 --publish never && pnpm run package:audit",
"build:mac:arm64": "pnpm run build && electron-builder --mac --arm64 --publish never && pnpm run package:audit",
"build:mac:release:x64": "pnpm run build && electron-builder --mac --x64 --publish never --config.forceCodeSigning=true && pnpm run package:audit",
"build:mac:release:arm64": "pnpm run build && electron-builder --mac --arm64 --publish never --config.forceCodeSigning=true && pnpm run package:audit",
"native:node": "node scripts/build/native/better-sqlite3.mjs ensure node",
"native:electron": "node scripts/build/native/better-sqlite3.mjs ensure electron",
@@ -23,6 +23,7 @@ LINK_PREVIEW_MAX_LINKS = 120
LINK_PREVIEW_TIMEOUT_SECONDS = 5
CDP_URL_ENV = "NCURATOR_WEB_CRAWLER_CDP_URL"
CDP_CONTEXT_MARKER_ENV = "NCURATOR_WEB_CRAWLER_CDP_CONTEXT_MARKER_URL"
CRAWLER_SESSION_ID = "ncurator-electron-cdp"
class CrawlFailure(Exception):
@@ -87,7 +88,9 @@ def to_jsonable(value: Any) -> Any:
if isinstance(value, (list, tuple, set)):
return [to_jsonable(item) for item in value]
if hasattr(value, "model_dump"):
return to_jsonable(value.model_dump(mode="json"))
# Crawl4AI can retain Python descriptors in reused result models. JSON
# mode rejects those descriptors before our recursive fallback can run.
return to_jsonable(value.model_dump(mode="python"))
if hasattr(value, "dict"):
return to_jsonable(value.dict())
if hasattr(value, "__dict__"):
@@ -196,6 +199,10 @@ def build_failure_data(
def build_crawler_run_config(topic: str) -> CrawlerRunConfig:
return CrawlerRunConfig(
# Electron's CDP context does not support creating a fresh target for
# every crawl. The sidecar processes requests serially, so one stable
# Crawl4AI session safely reuses the page across fetches.
session_id=CRAWLER_SESSION_ID,
markdown_generator=DefaultMarkdownGenerator(
content_filter=PruningContentFilter(
user_query="",
+3 -1
View File
@@ -206,7 +206,9 @@ export const assertSafePackagePaths = (paths) => {
}
}
const WEB_CRAWLER_SIDECAR_HARD_LIMIT_BYTES = 340 * 1024 * 1024
// Windows' frozen Python runtime is larger than macOS' runtime. Keep one
// cross-platform ceiling that still rejects the old bundled-browser layout.
const WEB_CRAWLER_SIDECAR_HARD_LIMIT_BYTES = 400 * 1024 * 1024
export const assertWebCrawlerSidecarPackage = (files, sourceLabel) => {
const sidecarFiles = files.filter((item) =>
+30 -143
View File
@@ -22,10 +22,7 @@ import { assertLivePublishAllowed, resolveReleaseStorageConfig } from './storage
const execFileAsync = promisify(execFile)
const ROOT = process.cwd()
const MACOS_PLATFORMS = ['macos-arm64', 'macos-x64']
const DEFAULT_E2E_PLATFORMS = ['macos-arm64']
const X64_E2E_DISABLED_REASON =
'macOS x64 package E2E is disabled by default because Rosetta execution is too slow; artifacts are still built, verified, signed, and published.'
const MACOS_PLATFORM = 'macos-arm64'
const stageDeadline = createReleaseStageDeadline('macOS release stage')
await loadReleaseEnvironment(ROOT)
const args = process.argv.slice(2)
@@ -34,7 +31,6 @@ const valueAfter = (name) => {
return index >= 0 ? args[index + 1] : undefined
}
const releaseTag = valueAfter('--tag') || ''
const includeX64E2E = args.includes('--include-x64-e2e')
const regression = readRegressionSelection(args)
const modes = [
['prepare', args.includes('--prepare-only')],
@@ -53,14 +49,11 @@ try {
}
if (!releaseVersion || modes.length !== 1) {
throw new Error(
'usage: node scripts/release/macos.mjs --tag <mac-v1.2.3> (--prepare-only|--test-only|--publish-only) [--include-x64-e2e] [--domain <name>] [--suite <name>] [--group <name>] [--case <caseId> ...] [--fresh-knowledge-cache]'
'usage: node scripts/release/macos.mjs --tag <mac-v1.2.3> (--prepare-only|--test-only|--publish-only) [--domain <name>] [--suite <name>] [--group <name>] [--case <caseId> ...] [--fresh-knowledge-cache]'
)
}
const mode = modes[0][0]
if (mode === 'publish') assertLivePublishAllowed()
if (includeX64E2E && mode !== 'test') {
throw new Error('--include-x64-e2e is only valid with --test-only')
}
if (mode !== 'test' && hasRegressionSelectionArgs(args)) {
throw new Error('regression selection options are only valid with --test-only')
}
@@ -82,13 +75,6 @@ const run = (command, commandArgs, options = {}) =>
})
})
const assertExecutable = async (filePath, label) => {
if (!filePath) throw new Error(`${label} is not configured`)
await access(filePath).catch(() => {
throw new Error(`${label} does not exist: ${filePath}`)
})
}
const validateTag = async () => {
const { stdout: tagType } = await execFileAsync('git', ['cat-file', '-t', releaseTag], {
cwd: ROOT
@@ -137,28 +123,6 @@ const validateEnvironment = async () => {
if (!identities.includes('Ncurator Update Signing')) {
throw new Error('Ncurator Update Signing is not installed in the macOS login keychain')
}
result.x64Node =
String(process.env.NCURATOR_X64_NODE || '').trim() ||
path.join(defaultToolRoot, 'node-x64', 'bin', 'node')
result.x64Python =
String(process.env.NCURATOR_X64_PYTHON || '').trim() ||
path.join(defaultToolRoot, 'python-x64', 'bin', 'python3')
await assertExecutable(result.x64Node, 'NCURATOR_X64_NODE')
await assertExecutable(result.x64Python, 'NCURATOR_X64_PYTHON')
const { stdout: nodeArch } = await execFileAsync('/usr/bin/arch', [
'-x86_64',
result.x64Node,
'-p',
'process.arch'
])
if (nodeArch.trim() !== 'x64') throw new Error('NCURATOR_X64_NODE is not x64')
const { stdout: pythonArch } = await execFileAsync('/usr/bin/arch', [
'-x86_64',
result.x64Python,
'-c',
'import platform; print(platform.machine())'
])
if (pythonArch.trim() !== 'x86_64') throw new Error('NCURATOR_X64_PYTHON is not x64')
}
return result
}
@@ -293,68 +257,27 @@ if (mode === 'prepare') {
keepWork: false
})
const archivePath = path.join(workspace.workDirectory, 'source.tar')
const armSource = path.join(workspace.workDirectory, 'arm64')
const x64Source = path.join(workspace.workDirectory, 'x64')
const sourceDirectory = path.join(workspace.workDirectory, 'arm64')
try {
await run('git', ['archive', '--format=tar', '--output', archivePath, releaseTag])
await extractTag(archivePath, armSource)
await extractTag(archivePath, x64Source)
const armEnvironment = {
await extractTag(archivePath, sourceDirectory)
const buildEnvironment = {
...process.env,
NCURATOR_SIDECAR_VENV_ROOT: tools.sidecarVenvRoot,
NCURATOR_SIDECAR_PIP_CACHE: path.join(tools.sidecarPipCacheRoot, 'darwin-arm64')
}
const x64Environment = {
...process.env,
PATH: `${path.dirname(tools.x64Node)}:${process.env.PATH || ''}`,
PIP_CERT: String(process.env.PIP_CERT || '').trim() || '/private/etc/ssl/cert.pem',
NCURATOR_SIDECAR_VENV_ROOT: tools.sidecarVenvRoot,
NCURATOR_SIDECAR_BUILD_PYTHON: tools.x64Python,
NCURATOR_SIDECAR_PIP_CACHE: path.join(tools.sidecarPipCacheRoot, 'darwin-x64')
}
await Promise.all([
(async () => {
await run(tools.pnpmPath, ['install', '--frozen-lockfile'], {
cwd: armSource,
env: armEnvironment
})
await seedNativeCache(armSource)
})(),
(async () => {
await run(
'/usr/bin/arch',
['-x86_64', tools.x64Node, tools.pnpmPath, 'install', '--frozen-lockfile'],
{ cwd: x64Source, env: x64Environment }
)
await seedNativeCache(x64Source)
})()
])
const [armCandidate, x64Candidate] = await Promise.all([
(async () => {
await run(tools.pnpmPath, ['run', 'build:mac:release:arm64'], {
cwd: armSource,
env: armEnvironment
})
return await createCandidate(armSource, 'macos-arm64', taggedCommit)
})(),
(async () => {
await run(
'/usr/bin/arch',
['-x86_64', tools.x64Node, tools.pnpmPath, 'run', 'build:mac:release:x64'],
{ cwd: x64Source, env: x64Environment }
)
return await createCandidate(x64Source, 'macos-x64', taggedCommit)
})()
])
await recordCandidateE2E({
tag: releaseTag,
platform: 'macos-x64',
status: 'skipped',
reason: X64_E2E_DISABLED_REASON,
selection: { mode: 'disabled-by-default' }
await run(tools.pnpmPath, ['install', '--frozen-lockfile'], {
cwd: sourceDirectory,
env: buildEnvironment
})
await seedNativeCache(sourceDirectory)
await run(tools.pnpmPath, ['run', 'build:mac:release:arm64'], {
cwd: sourceDirectory,
env: buildEnvironment
})
const candidate = await createCandidate(sourceDirectory, MACOS_PLATFORM, taggedCommit)
process.stdout.write(
`[release] macOS candidates prepared; arm64 package E2E is required before publish and x64 package E2E is disabled by default\n${armCandidate.bundlePath}\n${x64Candidate.bundlePath}\n`
`[release] macOS arm64 candidate prepared; package E2E is required before publish\n${candidate.bundlePath}\n`
)
} finally {
await workspace.dispose()
@@ -375,70 +298,34 @@ if (mode === 'test') {
await extractTag(archivePath, sourceDirectory)
await run(tools.pnpmPath, ['install', '--frozen-lockfile'], { cwd: sourceDirectory })
await seedNativeCache(sourceDirectory)
const e2ePlatforms = includeX64E2E ? MACOS_PLATFORMS : DEFAULT_E2E_PLATFORMS
if (!includeX64E2E) {
await recordCandidateE2E({
tag: releaseTag,
platform: 'macos-x64',
status: 'skipped',
reason: X64_E2E_DISABLED_REASON,
selection: { mode: 'disabled-by-default' }
})
const candidate = await readReleaseCandidate({ tag: releaseTag, platform: MACOS_PLATFORM })
if (candidate.state.commit !== taggedCommit) {
throw new Error(`${MACOS_PLATFORM} candidate commit does not match release tag`)
}
const candidates = await Promise.all(
e2ePlatforms.map(async (platform) => {
const candidate = await readReleaseCandidate({ tag: releaseTag, platform })
if (candidate.state.commit !== taggedCommit) {
throw new Error(`${platform} candidate commit does not match release tag`)
}
return candidate
})
)
await runCandidateE2E({ candidates, sourceDirectory, pnpmPath: tools.pnpmPath })
await runCandidateE2E({ candidates: [candidate], sourceDirectory, pnpmPath: tools.pnpmPath })
} finally {
await workspace.dispose()
}
process.stdout.write(
includeX64E2E
? `[release] macOS candidate E2E passed for both architectures\n`
: `[release] macOS arm64 candidate E2E passed; x64 package E2E is disabled by default\n`
)
process.stdout.write(`[release] macOS arm64 candidate E2E passed\n`)
process.exit(0)
}
await recordCandidateE2E({
const candidate = await assertCandidatePublishable({
tag: releaseTag,
platform: 'macos-x64',
status: 'skipped',
reason: X64_E2E_DISABLED_REASON,
selection: { mode: 'disabled-by-default' }
platform: MACOS_PLATFORM,
commit: taggedCommit
})
const candidates = await Promise.all(
MACOS_PLATFORMS.map(async (platform) =>
assertCandidatePublishable({ tag: releaseTag, platform, commit: taggedCommit })
)
)
if (
!candidates[0].verified.fragment.signature ||
candidates[0].verified.fragment.signature !== candidates[1].verified.fragment.signature
) {
throw new Error('macOS candidate signatures do not match')
}
for (const candidate of candidates) {
await run(process.execPath, [
path.join(ROOT, 'scripts', 'release', 'r2', 'upload-fragment.mjs'),
candidate.verified.directory,
candidate.verified.fragmentPath
])
}
await run(process.execPath, [
path.join(ROOT, 'scripts', 'release', 'r2', 'upload-fragment.mjs'),
candidate.verified.directory,
candidate.verified.fragmentPath
])
await run(process.execPath, [
path.join(ROOT, 'scripts', 'release', 'r2', 'finalize-macos.mjs'),
'--tag',
releaseTag
])
for (const platform of MACOS_PLATFORMS) {
await markCandidatePublished({ tag: releaseTag, platform })
}
await markCandidatePublished({ tag: releaseTag, platform: MACOS_PLATFORM })
try {
await cleanupLocalTestLogs(ROOT)
} catch (error) {
@@ -448,7 +335,7 @@ try {
try {
const removed = await cleanupSupersededReleaseCandidates({
tag: releaseTag,
platforms: ['macos-arm64', 'macos-x64']
platforms: [MACOS_PLATFORM]
})
if (removed.length > 0) {
process.stdout.write(`[release] removed ${removed.length} superseded local candidate(s)\n`)
-10
View File
@@ -25,16 +25,6 @@ const definitions = {
['Ncurator-mac-arm64.zip', 'Ncurator-mac-arm64.zip', 'update'],
['Ncurator-mac-arm64.zip.blockmap', 'Ncurator-mac-arm64.zip.blockmap', 'update']
]
},
'macos-x64': {
metadata: 'latest-mac.yml',
signature: 'mac-update-signature.txt',
assets: () => [
['Ncurator-mac-x64.dmg', 'Ncurator-mac-x64.dmg', 'installer'],
['Ncurator-mac-x64.dmg.blockmap', 'Ncurator-mac-x64.dmg.blockmap', 'update'],
['Ncurator-mac-x64.zip', 'Ncurator-mac-x64.zip', 'update'],
['Ncurator-mac-x64.zip.blockmap', 'Ncurator-mac-x64.zip.blockmap', 'update']
]
}
}
+67 -77
View File
@@ -51,24 +51,12 @@ const downloadAndVerifyFragment = async (platform) => {
}
try {
const [arm64, x64] = await Promise.all([
downloadAndVerifyFragment('macos-arm64'),
downloadAndVerifyFragment('macos-x64')
])
if (!arm64.signature || arm64.signature !== x64.signature) {
throw new Error('macOS arm64 and x64 release signatures do not match')
}
const arm64 = await downloadAndVerifyFragment('macos-arm64')
const metadataInput = path.join(workDirectory, 'metadata-input')
const metadataOutput = path.join(workDirectory, 'metadata-output')
await mkdir(path.join(metadataInput, 'mac-arm64'), { recursive: true })
await mkdir(path.join(metadataInput, 'mac-x64'), { recursive: true })
await writeFile(path.join(metadataInput, 'mac-arm64', 'latest-mac.yml'), arm64.metadata, 'utf8')
await writeFile(path.join(metadataInput, 'mac-x64', 'latest-mac.yml'), x64.metadata, 'utf8')
await execFileAsync(process.execPath, [
path.join(ROOT, 'scripts', 'release', 'r2', 'merge-mac-metadata.mjs'),
metadataInput
])
await mkdir(metadataInput, { recursive: true })
await writeFile(path.join(metadataInput, 'latest-mac.yml'), arm64.metadata, 'utf8')
await execFileAsync(process.execPath, [
path.join(ROOT, 'scripts', 'release', 'r2', 'metadata.mjs'),
metadataInput,
@@ -87,46 +75,36 @@ try {
'--no-progress'
])
const installers = new Map()
for (const fragment of [arm64, x64]) {
const installer = fragment.files.find((file) => file.role === 'installer')
const sourceKey = artifactKey(fragment, installer.storageName)
const downloadKey = fixedDownloadKey(fragment.platform)
const downloadName =
fragment.platform === 'macos-arm64'
? `Ncurator-${fragment.version}-macos-arm64.dmg`
: `Ncurator-${fragment.version}-macos-x64.dmg`
await aws([
's3api',
'copy-object',
'--bucket',
bucket,
'--copy-source',
`${bucket}/${sourceKey}`,
'--key',
downloadKey,
'--content-type',
'application/octet-stream',
'--cache-control',
'no-store',
'--content-disposition',
`attachment; filename="${downloadName}"`,
'--metadata-directive',
'REPLACE',
'--metadata',
`sha512=${installer.sha512}`
])
installers.set(fragment.platform, { installer, downloadKey, downloadName })
}
const installer = arm64.files.find((file) => file.role === 'installer')
const sourceKey = artifactKey(arm64, installer.storageName)
const downloadKey = fixedDownloadKey(arm64.platform)
const downloadName = `Ncurator-${arm64.version}-macos-arm64.dmg`
await aws([
's3api',
'copy-object',
'--bucket',
bucket,
'--copy-source',
`${bucket}/${sourceKey}`,
'--key',
downloadKey,
'--content-type',
'application/octet-stream',
'--cache-control',
'no-store',
'--content-disposition',
`attachment; filename="${downloadName}"`,
'--metadata-directive',
'REPLACE',
'--metadata',
`sha512=${installer.sha512}`
])
const toDownload = (platform) => {
const { installer, downloadKey, downloadName } = installers.get(platform)
return {
name: downloadName,
url: `${publicBaseUrl}/${downloadKey}`,
size: installer.size,
sha512: installer.sha512
}
const download = {
name: downloadName,
url: `${publicBaseUrl}/${downloadKey}`,
size: installer.size,
sha512: installer.sha512
}
const manifest = {
schemaVersion: 2,
@@ -134,7 +112,7 @@ try {
version: arm64.version,
tag: arm64.tag,
publishedAt: new Date().toISOString(),
downloads: { arm64: toDownload('macos-arm64'), x64: toDownload('macos-x64') }
downloads: { arm64: download }
}
const manifestPath = path.join(workDirectory, 'latest.json')
await writeFile(manifestPath, `${JSON.stringify(manifest, null, 2)}\n`, 'utf8')
@@ -157,47 +135,59 @@ try {
const metadata = metadataResponse.ok ? await metadataResponse.text() : ''
if (
!metadata.includes(`version: ${releaseVersion}`) ||
!metadata.includes(`artifacts/macos/${releaseTag}/Ncurator-mac-arm64.zip`) ||
!metadata.includes(`artifacts/macos/${releaseTag}/Ncurator-mac-x64.zip`)
!metadata.includes(`artifacts/macos/${releaseTag}/Ncurator-mac-arm64.zip`)
) {
throw new Error('public latest-mac.yml does not match the Mac release')
}
for (const { installer, downloadKey, downloadName } of installers.values()) {
const response = await fetchWithRetry(`${publicBaseUrl}/${downloadKey}?check=${Date.now()}`, {
method: 'HEAD',
cache: 'no-store'
})
if (
!response.ok ||
Number(response.headers.get('content-length')) !== installer.size ||
!String(response.headers.get('content-disposition') || '').includes(
`filename="${downloadName}"`
)
) {
throw new Error(`public fixed macOS download verification failed: ${downloadKey}`)
}
const response = await fetchWithRetry(`${publicBaseUrl}/${downloadKey}?check=${Date.now()}`, {
method: 'HEAD',
cache: 'no-store'
})
if (
!response.ok ||
Number(response.headers.get('content-length')) !== installer.size ||
!String(response.headers.get('content-disposition') || '').includes(
`filename="${downloadName}"`
)
) {
throw new Error(`public fixed macOS download verification failed: ${downloadKey}`)
}
const removable = (await listKeys('artifacts/macos/')).filter(
(key) => !key.startsWith(`artifacts/macos/${releaseTag}/`)
)
const activeArtifactKeys = new Set(
arm64.files.map((file) => artifactKey(arm64, file.storageName))
)
removable.push(
...(await listKeys(`artifacts/macos/${releaseTag}/`)).filter(
(key) => !activeArtifactKeys.has(key)
)
)
removable.push(
...(await listKeys('release-status/macos/')).filter(
(key) => !key.startsWith(`release-status/macos/${releaseTag}/`)
)
)
const activeStatusKey = statusKey(releaseTag, arm64.platform)
removable.push(
...(await listKeys(`release-status/macos/${releaseTag}/`)).filter(
(key) => key !== activeStatusKey
)
)
removable.push(
...(await listKeys('downloads/latest/')).filter(
(key) => key.startsWith('downloads/latest/macos-') && key !== downloadKey
)
)
removable.push(
...(await listKeys('artifacts/v')).filter((key) => /\/Ncurator-mac-[^/]+$/u.test(key))
)
removable.push(
...(await listKeys('release-status/v')).filter((key) =>
/\/macos-(?:arm64|x64)\.json$/u.test(key)
)
...(await listKeys('release-status/v')).filter((key) => /\/macos-arm64\.json$/u.test(key))
)
await deleteKeys(removable)
process.stdout.write(
`[release] macOS ${releaseTag} is live on independent fixed download channels\n`
)
process.stdout.write(`[release] macOS arm64 ${releaseTag} is live\n`)
} finally {
await rm(workDirectory, { recursive: true, force: true })
}
+3 -3
View File
@@ -2,8 +2,7 @@ import { parseReleaseTag } from '../tag.mjs'
const PLATFORM_AUDIT = Object.freeze({
'windows-x64': { platform: 'win32', arch: 'x64', requiresSignature: false },
'macos-arm64': { platform: 'darwin', arch: 'arm64', requiresSignature: true },
'macos-x64': { platform: 'darwin', arch: 'x64', requiresSignature: true }
'macos-arm64': { platform: 'darwin', arch: 'arm64', requiresSignature: true }
})
const SHA512_BASE64 = /^[A-Za-z0-9+/]{86}==$/u
@@ -105,7 +104,8 @@ export const assertReleaseFragment = (
if (!isNonEmptyString(file.storageName) || !STORAGE_NAME.test(file.storageName)) {
fail(label, `storageName must be an ASCII filename for ${file.name}`)
}
if (storageNames.has(file.storageName)) fail(label, `duplicate storage asset ${file.storageName}`)
if (storageNames.has(file.storageName))
fail(label, `duplicate storage asset ${file.storageName}`)
storageNames.add(file.storageName)
if (!['installer', 'update'].includes(file.role)) fail(label, `invalid role for ${file.name}`)
if (!isPositiveInteger(file.size)) fail(label, `invalid size for ${file.name}`)
-117
View File
@@ -1,117 +0,0 @@
import fs from 'node:fs'
import path from 'node:path'
import process from 'node:process'
const root = path.resolve(process.argv[2] || 'release-artifacts')
const outputPath = path.join(root, 'latest-mac.yml')
const walk = (directory) => {
const entries = fs.readdirSync(directory, { withFileTypes: true })
return entries.flatMap((entry) => {
const entryPath = path.join(directory, entry.name)
if (entry.isDirectory()) return walk(entryPath)
return [entryPath]
})
}
const parseScalar = (content, key) => {
const match = content.match(new RegExp(`^${key}:\\s*(.+)$`, 'm'))
if (!match) return ''
return match[1].trim().replace(/^'|'$/g, '')
}
const parseFileEntries = (content) => {
const entries = []
let current = null
for (const line of content.split('\n')) {
const urlMatch = line.match(/^\s{2}- url:\s*(.+)$/)
if (urlMatch) {
current = {
url: urlMatch[1].trim(),
sha512: '',
size: 0
}
entries.push(current)
continue
}
if (!current) continue
const sha512Match = line.match(/^\s{4}sha512:\s*(.+)$/)
if (sha512Match) {
current.sha512 = sha512Match[1].trim()
continue
}
const sizeMatch = line.match(/^\s{4}size:\s*(\d+)$/)
if (sizeMatch) {
current.size = Number(sizeMatch[1])
}
}
return entries.filter((entry) => entry.url && entry.sha512 && entry.size > 0)
}
const yamlFiles = walk(root).filter((filePath) => path.basename(filePath) === 'latest-mac.yml')
if (yamlFiles.length === 0) {
process.stderr.write(`No latest-mac.yml files found under ${root}\n`)
process.exit(1)
}
const parsedFiles = yamlFiles.map((filePath) => {
const content = fs.readFileSync(filePath, 'utf8')
return {
filePath,
version: parseScalar(content, 'version'),
releaseDate: parseScalar(content, 'releaseDate'),
entries: parseFileEntries(content)
}
})
const version = parsedFiles[0].version
if (!version || parsedFiles.some((item) => item.version !== version)) {
process.stderr.write('latest-mac.yml files must have the same non-empty version\n')
process.exit(1)
}
const entriesByUrl = new Map()
for (const item of parsedFiles) {
for (const entry of item.entries) {
entriesByUrl.set(entry.url, entry)
}
}
const entries = [...entriesByUrl.values()].sort((left, right) => left.url.localeCompare(right.url))
const primaryEntry =
entries.find((entry) => entry.url.endsWith('.zip') && entry.url.includes('arm64')) ||
entries.find((entry) => entry.url.endsWith('.zip')) ||
entries[0]
if (!primaryEntry) {
process.stderr.write('No macOS release files found in latest-mac.yml files\n')
process.exit(1)
}
for (const filePath of yamlFiles) {
fs.unlinkSync(filePath)
}
const releaseDate =
parsedFiles.find((item) => item.releaseDate)?.releaseDate || new Date().toISOString()
const lines = [
`version: ${version}`,
'files:',
...entries.flatMap((entry) => [
` - url: ${entry.url}`,
` sha512: ${entry.sha512}`,
` size: ${entry.size}`
]),
`path: ${primaryEntry.url}`,
`sha512: ${primaryEntry.sha512}`,
`releaseDate: '${releaseDate}'`,
''
]
fs.writeFileSync(outputPath, lines.join('\n'), 'utf8')
+1 -2
View File
@@ -1,6 +1,6 @@
export const platformFamily = (platform) => {
if (platform === 'windows-x64') return 'windows'
if (platform === 'macos-arm64' || platform === 'macos-x64') return 'macos'
if (platform === 'macos-arm64') return 'macos'
throw new Error(`unsupported release platform: ${platform}`)
}
@@ -18,7 +18,6 @@ export const fixedDownloadKey = (platform, role = 'installer') => {
if (role !== 'installer') throw new Error(`unsupported fixed download role: ${role}`)
if (platform === 'windows-x64') return 'downloads/latest/windows-x64.exe'
if (platform === 'macos-arm64') return 'downloads/latest/macos-arm64.dmg'
if (platform === 'macos-x64') return 'downloads/latest/macos-x64.dmg'
throw new Error(`unsupported release platform: ${platform}`)
}
+1 -1
View File
@@ -29,7 +29,7 @@ try {
} catch {
// Public status checks can still run against the domestic channel without local credentials.
}
const platforms = releaseFamily === 'mac' ? ['macos-arm64', 'macos-x64'] : ['windows-x64']
const platforms = releaseFamily === 'mac' ? ['macos-arm64'] : ['windows-x64']
const awsExecutable = await resolveAwsExecutable()
if (!awsExecutable) throw new Error('AWS CLI is required to verify release sha512 metadata')
for (const platform of platforms) {
+1 -1
View File
@@ -2,7 +2,7 @@ export const RELEASE_VERSION_PATTERN = /^\d+\.\d+\.\d+$/u
export const RELEASE_TAG_PATTERN = /^(mac|windows)-v(\d+\.\d+\.\d+)$/u
const normalizeFamily = (platform) => {
if (platform === 'mac' || platform === 'macos-arm64' || platform === 'macos-x64') return 'mac'
if (platform === 'mac' || platform === 'macos-arm64') return 'mac'
if (platform === 'windows' || platform === 'windows-x64') return 'windows'
throw new Error(`unsupported release platform: ${String(platform)}`)
}
+2 -29
View File
@@ -28,13 +28,11 @@ const BETTER_SQLITE_NATIVE_BINDING_ENV = 'NCURATOR_BETTER_SQLITE3_NATIVE_BINDING
const LIVE_BACKEND_NODE_OLD_SPACE_MB = 8192
const BACKEND_WORKERS = 6
const E2E_WORKERS = 3
const E2E_WORKERS_BY_TARGET = new Map([['macos-x64', 1]])
const LIVE_WORKERS = 3
const RUNNER_DEADLINE_MS = 55 * 60 * 1000
const DEADLINE_EXIT_CODE = 124
const STALE_E2E_TEMP_MAX_AGE_MS = 24 * 60 * 60 * 1000
const E2E_EXECUTION_CLASSES = new Set(['normal', 'live', 'exclusive', 'warmup'])
const CROSS_ARCH_WARMUP_SOURCE_TARGET = new Map([['macos-x64', 'macos-arm64']])
const USE_WINDOWS_SHELL = process.platform === 'win32'
const activeAsyncChildren = new Set()
let stoppingActiveChildren = false
@@ -330,17 +328,6 @@ const seedCaseKnowledgeCache = async (caseEnv, seedPath) => {
})
}
const resolveWarmupSeedPath = (task, warmupCacheByTarget) => {
const explicitSeedPath = String(
task.env.NCURATOR_E2E_CROSS_ARCH_WARMUP_SEED_PATH ||
process.env.NCURATOR_E2E_CROSS_ARCH_WARMUP_SEED_PATH ||
''
).trim()
if (explicitSeedPath) return explicitSeedPath
const sourceTargetId = CROSS_ARCH_WARMUP_SOURCE_TARGET.get(task.targetId)
return sourceTargetId ? String(warmupCacheByTarget.get(sourceTargetId) || '').trim() : ''
}
const fileContainsCaseId = async (filePath, caseId) => {
if (toCaseId(filePath) === caseId) return true
const source = await readFile(filePath, 'utf8')
@@ -778,21 +765,7 @@ const runE2ETestFilesConcurrently = async (testFiles, targets, testNamePattern,
process.stdout.write(
`[test-workflow] vector warm-up ${String(index + 1)}/${String(warmupTasks.length)} must pass before parallel E2E starts\n`
)
const sourceTargetId = CROSS_ARCH_WARMUP_SOURCE_TARGET.get(task.targetId)
const warmupSeedPath = resolveWarmupSeedPath(task, warmupCacheByTarget)
if (sourceTargetId && !warmupSeedPath) {
throw new Error(
`${task.targetId} vector warm-up requires a passed ${sourceTargetId} cache seed; ` +
'run both package targets in release order or set ' +
'NCURATOR_E2E_CROSS_ARCH_WARMUP_SEED_PATH'
)
}
if (warmupSeedPath) {
process.stdout.write(
`[test-workflow] ${task.targetId} vector warm-up uses verified cross-architecture cache seed\n`
)
}
const outcome = await runTask(task, warmupSeedPath)
const outcome = await runTask(task)
if (outcome.code !== 0) {
exitCode = outcome.code
process.stderr.write(
@@ -818,7 +791,7 @@ const runE2ETestFilesConcurrently = async (testFiles, targets, testNamePattern,
while (pending.length > 0 && active.size < E2E_WORKERS && Date.now() < deadlineAt) {
const nextIndex = pending.findIndex((task) => {
if (task.execution === 'live' && liveActive >= LIVE_WORKERS) return false
const targetLimit = E2E_WORKERS_BY_TARGET.get(task.targetId) || E2E_WORKERS
const targetLimit = E2E_WORKERS
return Number(activeByTarget.get(task.targetId) || 0) < targetLimit
})
if (nextIndex < 0) break
@@ -1,4 +1,5 @@
import { AIMessage, type BaseMessage } from '@langchain/core/messages'
import { normalizeThoughtText } from '../../../../../../shared/thoughtText'
const isProcessMessage = (message: BaseMessage): boolean => {
const type = message.getType()
@@ -42,11 +43,15 @@ const formatToolCallArgs = (args: Record<string, unknown>): string =>
.map(([key, value]) => `${key}=${formatToolCallArgValue(value)}`)
.join(', ')
const normalizeCompactionText = (content: string): string => content.trim()
const renderAiMessageLines = (message: AIMessage): string[] => {
const lines: string[] = []
const content =
const rawContent =
typeof message.content === 'string' ? message.content : JSON.stringify(message.content)
if (content.trim()) {
const content =
typeof message.content === 'string' ? normalizeThoughtText(rawContent) : normalizeCompactionText(rawContent)
if (content) {
lines.push(`[ai] ${content}`)
}
const toolCalls = message.tool_calls
@@ -61,9 +66,9 @@ const renderAiMessageLines = (message: AIMessage): string[] => {
}
const renderToolMessageLine = (message: BaseMessage): string => {
const content =
const rawContent =
typeof message.content === 'string' ? message.content : JSON.stringify(message.content)
return `[tool] ${content}`
return `[tool] ${normalizeCompactionText(rawContent)}`
}
// 每个 AI 消息及其触发的工具调用、紧随的工具结果为一轮,轮首加 [round N]。
@@ -8,9 +8,9 @@ import {
loadContextCompactionWorkspaceSeed
} from '../../../../harness/backend/liveScenarios/contextCompactionScenario'
import {
collectSuccessfulDocxWriteSourcePaths,
filterAiRunEventsByType,
hasDocxApplyArtifact,
hasDocxFileArtifact,
hasDocxDeliveryEvidence,
isDocxContentCreateInput,
runWithStructuredOutputRetry
} from '../testUtils'
@@ -112,6 +112,10 @@ describe('ai human live backend', () => {
)
.map((event) => event.input.sourcePath)
.filter((value): value is string => typeof value === 'string')
const docxWriteSourcePaths = collectSuccessfulDocxWriteSourcePaths(
events,
successfulToolCallIds
)
const textCreateSourcePaths = filterAiRunEventsByType(events, 'tool_call_started')
.filter(
(event) =>
@@ -181,10 +185,13 @@ describe('ai human live backend', () => {
(toolNames.some((toolName) => toolName.startsWith('docx_')) ||
toolNames.includes('text_file_edit'))
const artifactSatisfied =
(hasDocxFileArtifact(result.output?.artifacts) &&
hasDocxApplyArtifact(result.output?.artifacts) &&
createSourcePaths.length >= 1) ||
textCreateSourcePaths.length >= 1
hasDocxDeliveryEvidence({
artifacts: result.output?.artifacts,
events
}) &&
(docxWriteSourcePaths.length >= 1 ||
createSourcePaths.length >= 1 ||
textCreateSourcePaths.length >= 1)
runtime.assertThat(
lifecycleSatisfied &&
summaryFormatSatisfied &&
@@ -206,6 +213,7 @@ describe('ai human live backend', () => {
ratios,
toolNames,
createSourcePaths,
docxWriteSourcePaths,
textCreateSourcePaths,
artifacts: result.output?.artifacts
}
@@ -23,7 +23,8 @@ describe('ai human live backend', () => {
domain: 'ai',
suite: 'human',
groups: ['live'],
description: '用户给纯文本诉求,AI 提炼搜索意图后走 web_search(type=search/fetch) 读官方文档回答。',
description:
'用户给纯文本诉求,AI 提炼搜索意图后走 web_search(type=search/fetch) 读官方文档回答。',
fixtures: [],
runMode: 'single',
runtimeModelName: loadLiveLlmConfigFromEnv().model_name,
@@ -65,16 +66,21 @@ describe('ai human live backend', () => {
return Boolean(input)
})
.map((event) => {
const data = event.result?.data
if (!data || typeof data !== 'object' || Array.isArray(data)) return ''
const value = data as { finalUrl?: unknown; requestedUrl?: unknown }
return String(value.finalUrl || value.requestedUrl || '')
})
const data = event.result?.data
if (!data || typeof data !== 'object' || Array.isArray(data)) return ''
const value = data as { finalUrl?: unknown; requestedUrl?: unknown }
return String(value.finalUrl || value.requestedUrl || '')
})
const officialUrls = fetchUrls.filter(isPandasHost)
const readCsvFetched = officialUrls.some((url) => url.includes('/pandas.read_csv.html'))
const toCsvFetched = officialUrls.some((url) =>
url.includes('/pandas.DataFrame.to_csv.html')
)
runtime.assertThat(
result.output.status === 'completed' &&
searchCalls.length >= 1 &&
officialUrls.length >= 1,
readCsvFetched &&
toCsvFetched,
{
name: 'human-live-web-search-intent-query-reads-official-sources',
message:
@@ -83,7 +89,9 @@ describe('ai human live backend', () => {
outputStatus: result.output?.status,
searchCalls: searchCalls.map((event) => event.input?.query),
fetchUrls,
officialUrls
officialUrls,
readCsvFetched,
toCsvFetched
}
}
)
+29
View File
@@ -156,6 +156,35 @@ export const hasDocxFileArtifact = (artifacts: readonly AiArtifactRef[] | undefi
Array.isArray(artifacts) &&
artifacts.some((artifact) => artifact.kind === 'file' && artifact.capability === 'docx')
export const hasDocxApplyArtifactEvent = (events: readonly AiRunEvent[]): boolean =>
filterAiRunEventsByType(events, 'file_artifact').some(
(event) =>
event.capability === 'docx' &&
typeof event.applyId === 'string' &&
event.applyId.length > 0
)
export const hasDocxDeliveryEvidence = (input: {
artifacts?: readonly AiArtifactRef[]
events: readonly AiRunEvent[]
}): boolean =>
(hasDocxFileArtifact(input.artifacts) && hasDocxApplyArtifact(input.artifacts)) ||
hasDocxApplyArtifactEvent(input.events)
export const collectSuccessfulDocxWriteSourcePaths = (
events: readonly AiRunEvent[],
successfulToolCallIds: ReadonlySet<string>
): string[] =>
filterAiRunEventsByType(events, 'tool_call_started')
.filter(
(event) =>
DOCX_WRITE_TOOL_NAMES.has(event.toolName) &&
successfulToolCallIds.has(event.toolCallId) &&
isDocxWriteInput(event.input)
)
.map((event) => event.input.sourcePath)
.filter((value): value is string => typeof value === 'string')
export const countDocxFileArtifacts = (artifacts: readonly AiArtifactRef[] | undefined): number =>
Array.isArray(artifacts)
? artifacts.filter((artifact) => artifact.kind === 'file' && artifact.capability === 'docx')
@@ -294,6 +294,53 @@ describe('ai tool web backend', () => {
})
})
it('web-search-fetch-reuses-one-sidecar-for-multiple-pages', async () => {
await runBackendCase({
caseId: 'web-search-fetch-reuses-one-sidecar-for-multiple-pages',
harnessMode: 'tool-real',
domain: 'ai',
suite: 'tool',
groups: ['web-browsing'],
description: '验证同一个 sidecar 和 Crawl4AI 实例可以连续抓取多个页面。',
fixtures: [],
runMode: 'single',
execute: async (runtime) => {
const server = await createLocalServer()
const webCrawlerService = await createWebCrawlerService()
try {
const harness = createWebToolHarness(runtime, { webCrawlerService })
const policy = await harness.invoke<WebSearchFetchPayload>('web_search', {
type: 'fetch',
url: `${server.baseUrl}/policy`
})
const guide = await harness.invoke<WebSearchFetchPayload>('web_search', {
type: 'fetch',
url: `${server.baseUrl}/guide`
})
return { policy, guide }
} finally {
await webCrawlerService.close()
await server.close()
}
},
assert: async (result, runtime) => {
runtime.assertThat(
!result.policy.error &&
!result.guide.error &&
result.policy.finalUrl?.endsWith('/policy') === true &&
result.guide.finalUrl?.endsWith('/guide') === true &&
result.policy.markdown?.includes('申报截止时间为 2026-09-30') === true &&
result.guide.markdown?.includes('办理材料包括身份证') === true,
{
name: 'web-search-fetch-reuses-sidecar-without-serialization-failure',
message: '连续抓取应复用同一个 sidecar,并且两个页面都返回可用正文。',
data: result
}
)
}
})
})
it('web-search-fetch-http-failure', async () => {
await runBackendCase({
caseId: 'web-search-fetch-http-failure',
@@ -328,7 +375,8 @@ describe('ai tool web backend', () => {
result.finalUrl?.endsWith('/missing') === true,
{
name: 'web-search-fetch-preserves-http-failure-fields',
message: 'web_search type=fetch HTTP 失败时应保留错误文本、requestedUrl/finalUrl 和状态码。',
message:
'web_search type=fetch HTTP 失败时应保留错误文本、requestedUrl/finalUrl 和状态码。',
data: result
}
)
+26
View File
@@ -99,6 +99,32 @@ describe('renderMessagesForCompaction', () => {
)
})
it('trims leading and trailing whitespace from AI thoughts and tool results', () => {
const rendered = renderMessagesForCompaction([
new AIMessage({
content: '材料齐了。\n\n',
tool_calls: [toolCall('1', 'terminal_run', { command: 'ls' })]
}),
new ToolMessage({ content: '\n\ntotal 1\n', tool_call_id: '1' })
])
expect(rendered).toBe(
'[round 1]\n[ai] 材料齐了。\n[tool_call] terminal_run(command="ls")\n[tool] total 1'
)
})
it('preserves internal blank lines in tool output', () => {
const rendered = renderMessagesForCompaction([
new AIMessage({
content: '',
tool_calls: [toolCall('1', 'terminal_run', { command: 'ls' })]
}),
new ToolMessage({ content: '\nline 1\n\nline 3\n', tool_call_id: '1' })
])
expect(rendered).toBe(
'[round 1]\n[tool_call] terminal_run(command="ls")\n[tool] line 1\n\nline 3'
)
})
it('json-stringifies nested args and truncates long values', () => {
const long = 'x'.repeat(400)
const rendered = renderMessagesForCompaction([
@@ -157,21 +157,18 @@ describe('app update publish config', () => {
expect(testWorkflow).toContain('vector warm-up failed; parallel E2E will not start')
expect(testWorkflow).toContain('seedCaseKnowledgeCache')
expect(testWorkflow).toContain("path.basename(source) !== 'stats.json'")
expect(testWorkflow).toContain("new Map([['macos-x64', 'macos-arm64']])")
expect(testWorkflow).toContain("E2E_WORKERS_BY_TARGET = new Map([['macos-x64', 1]])")
expect(testWorkflow).not.toContain('macos-x64')
expect(testWorkflow).toContain('const LIVE_WORKERS = 3')
expect(testWorkflow).not.toContain('LIVE_BACKEND_RETRIES')
expect(testWorkflow).not.toContain("extraArgs.push('--retry'")
expect(testWorkflow).toContain('parsed.caseIds.push(next)')
expect(testWorkflow).toContain('no test files match --case')
expect(testWorkflow).toContain('NCURATOR_TEST_RUNNER_DEADLINE_MS')
expect(testWorkflow).toContain('NCURATOR_E2E_CROSS_ARCH_WARMUP_SEED_PATH')
expect(testWorkflow).toContain('uses verified cross-architecture cache seed')
expect(testWorkflow).toContain('targetTests.every')
expect(macRelease).toContain("if (result.status === 'passed') status = 'passed'")
expect(macRelease).toContain("const DEFAULT_E2E_PLATFORMS = ['macos-arm64']")
expect(macRelease).toContain("args.includes('--include-x64-e2e')")
expect(macRelease).toContain("status: 'skipped'")
expect(macRelease).toContain("const MACOS_PLATFORM = 'macos-arm64'")
expect(macRelease).not.toContain('macos-x64')
expect(macRelease).not.toContain('--include-x64-e2e')
expect(testWorkflow).toContain("spawnSync('taskkill.exe'")
expect(testWorkflow).toContain("input.type === 'e2e' && input.target === 'package'")
expect(testWorkflow).toContain("prepareBetterSqliteNative('node')")
@@ -45,7 +45,7 @@ describe('package size budget', () => {
expect(result).toEqual({ status: 'not-configured', platform: 'win32-x64', metrics: [] })
})
it('loads the approved macOS arm64 and x64 baselines', () => {
it('loads only the approved macOS arm64 baseline', () => {
const config = loadPackageSizeBudgetConfig(path.resolve('build/package-size-budgets.json'))
expect(config.platforms['darwin-arm64']).toMatchObject({
@@ -54,11 +54,6 @@ describe('package size budget', () => {
dmg: { baselineMiB: 432.9, hardLimitMiB: 480 },
zip: { baselineMiB: 418.5, hardLimitMiB: 465 }
})
expect(config.platforms['darwin-x64']).toMatchObject({
installedApp: { baselineMiB: 1145.6, hardLimitMiB: 1230 },
appAsar: { baselineMiB: 230.9, hardLimitMiB: 252 },
dmg: { baselineMiB: 447.9, hardLimitMiB: 495 },
zip: { baselineMiB: 433.3, hardLimitMiB: 480 }
})
expect(config.platforms['darwin-x64']).toBeUndefined()
})
})
@@ -27,6 +27,7 @@ describe('release fragment', () => {
expect(createReleaseTag('mac', '0.1.30')).toBe('mac-v0.1.30')
expect(createReleaseTag('windows', '0.1.32')).toBe('windows-v0.1.32')
expect(parseReleaseTag('mac-v0.1.30', 'macos-arm64').version).toBe('0.1.30')
expect(() => parseReleaseTag('mac-v0.1.30', 'macos-x64')).toThrow(/unsupported/u)
expect(() => parseReleaseTag('windows-v0.1.32', 'macos-arm64')).toThrow(/does not match/u)
expect(compareReleaseVersions('0.1.30', '0.1.29')).toBeGreaterThan(0)
expect(compareReleaseVersions('0.1.30', '0.1.31')).toBeLessThan(0)
@@ -16,14 +16,14 @@ describe('release storage layout', () => {
expect(statusKey('windows-v0.2.0', 'windows-x64')).toBe(
'release-status/windows/windows-v0.2.0.json'
)
expect(statusKey('mac-v0.1.6', 'macos-x64')).toBe(
'release-status/macos/mac-v0.1.6/macos-x64.json'
expect(statusKey('mac-v0.1.6', 'macos-arm64')).toBe(
'release-status/macos/mac-v0.1.6/macos-arm64.json'
)
})
it('uses stable direct-download object keys', () => {
expect(fixedDownloadKey('windows-x64')).toBe('downloads/latest/windows-x64.exe')
expect(fixedDownloadKey('macos-arm64')).toBe('downloads/latest/macos-arm64.dmg')
expect(fixedDownloadKey('macos-x64')).toBe('downloads/latest/macos-x64.dmg')
expect(() => fixedDownloadKey('macos-x64')).toThrow(/unsupported release platform/u)
})
})