diff --git a/.gitignore b/.gitignore index efdbdaf0..95e20011 100644 --- a/.gitignore +++ b/.gitignore @@ -2,6 +2,7 @@ node_modules dist out .env +.env.* .DS_Store .eslintcache *.tsbuildinfo diff --git a/README.en.md b/README.en.md index bfd429f9..9fdea1fc 100644 --- a/README.en.md +++ b/README.en.md @@ -6,8 +6,6 @@

Building the most user-friendly AI assistant for local office documents

-

Read your files. Edit your documents. Review the changes. Get work done.

-

简体中文 · English

@@ -49,18 +47,17 @@ See the [website](https://www.guanzhangai.cn/zh) for demos and real task example 2. Choose an available model or configure your own endpoint and API key. 3. Create a partner, add a working folder, and give it a task. To use the knowledge base, download an embedding model and build an index first. -The app is free; model costs depend on the service you choose. Files and indexes are stored locally. Calls to external models or network services send relevant content as needed for the task. - ## Roadmap Join the discussion and help shape these plans: -| Direction | Planned work | -| ---------------------------- | -------------------------------------------------------------------------- | -| 🌍 Chinese and English | Chinese and English UI, plus evaluation of English system and tool prompts | -| 🧩 Skill integration | Connect and manage custom and third-party Skills | -| 🧭 Browser data import | Import data from your browser, starting with bookmarks and history | -| 🖼️ Multimodal knowledge base | Parse, index, and retrieve images and audio with source references | +| Direction | Planned work | +| ---------------------------- | ------------------------------------------------------------------ | +| 🌍 English support | An English user interface | +| 🔤 English knowledge base | English embedding models and indexing | +| 🧩 Skill integration | Connect and manage custom and third-party Skills | +| 🧭 Browser data import | Import data from your browser, starting with bookmarks and history | +| 🖼️ Multimodal knowledge base | Parse, index, and retrieve images and audio with source references | ## Get involved @@ -83,7 +80,7 @@ See the [contribution guide](./CONTRIBUTING.en.md#development-and-validation) fo ## Browser extension -A separately maintained bookmark viewer and URL copying tool: [Chrome](https://chromewebstore.google.com/detail/ncurator-knowledge-base-a/jdlfflbeekclkgcckcfombmfojmcdeio?hl=zh-CN) · [Microsoft Edge](https://microsoftedge.microsoft.com/addons/detail/hkdfdbpkmkpooopbkdghecbaipeoijpj). +A separately maintained bookmark viewer and URL copying tool: [Source repository](https://github.com/Yoan98/Ncurator-Extension) · [Chrome](https://chromewebstore.google.com/detail/ncurator-knowledge-base-a/jdlfflbeekclkgcckcfombmfojmcdeio?hl=zh-CN) · [Microsoft Edge](https://microsoftedge.microsoft.com/addons/detail/hkdfdbpkmkpooopbkdghecbaipeoijpj). ## License diff --git a/README.md b/README.md index e4148a44..62a25d75 100644 --- a/README.md +++ b/README.md @@ -80,7 +80,7 @@ pnpm dev ## 浏览器插件 -独立维护的收藏夹查看与链接复制工具:[Chrome](https://chromewebstore.google.com/detail/ncurator-knowledge-base-a/jdlfflbeekclkgcckcfombmfojmcdeio?hl=zh-CN) · [Microsoft Edge](https://microsoftedge.microsoft.com/addons/detail/hkdfdbpkmkpooopbkdghecbaipeoijpj)。 +独立维护的收藏夹查看与链接复制工具:[源码仓库](https://github.com/Yoan98/Ncurator-Extension) · [Chrome](https://chromewebstore.google.com/detail/ncurator-knowledge-base-a/jdlfflbeekclkgcckcfombmfojmcdeio?hl=zh-CN) · [Microsoft Edge](https://microsoftedge.microsoft.com/addons/detail/hkdfdbpkmkpooopbkdghecbaipeoijpj)。 ## 许可证 diff --git a/doc/deploy/design.md b/doc/deploy/design.md index 61c3c7a8..6c706d1d 100644 --- a/doc/deploy/design.md +++ b/doc/deploy/design.md @@ -4,11 +4,11 @@ ### Windows -本地 Windows x64:Node.js 22 x64、项目指定 pnpm、Python 3.12 x64、Git、tar;发 COS 时还需 AWS CLI 与 COS 存储配置。仅未来海外版才需要 R2 配置。`pnpm release:doctor` 检查本机条件。prepare / test / publish 只消费本机候选包。 +本地 Windows x64:Node.js 22 x64、项目指定 pnpm、Python 3.12 x64、Git、tar;发布时还需 AWS CLI、COS 存储配置,以及已登录且具备公开仓库写权限的 GitHub CLI。仅未来海外版才需要 R2 配置。`pnpm release:doctor` 检查本机条件。prepare / test / publish 只消费本机候选包。 ### macOS -prepare 必须在 Apple Silicon Mac;需已信任的 `Ncurator Update Signing` P12(自建更新签名,`timestamp: none`、`notarize: false`,非 Apple Developer ID 公证)。候选包须完成正式包 E2E。不构建、测试或发布 Intel Mac。 +prepare 必须在 Apple Silicon Mac;需已信任的 `Ncurator Update Signing` P12(自建更新签名,`timestamp: none`、`notarize: false`,非 Apple Developer ID 公证)。候选包须完成正式包 E2E;发布时需 AWS CLI、COS 存储配置及已登录的 GitHub CLI。不构建、测试或发布 Intel Mac。 ### 构建与审计硬约束 @@ -43,15 +43,15 @@ CDN 节点缓存遵循源站 `Cache-Control`:安装包为 `public, max-age=315 ### GitHub Releases 手动下载 -公开仓库 `Yoan98/Ncurator` 的 Releases 作为安装包手动下载镜像,上传与 COS 上线版本相同、已完成正式包回归的文件,不重新构建。Windows 与 macOS 沿用各自独立 Tag,每条 Release 包含对应平台的 EXE 或 DMG、`SHA256SUMS` 和版本说明;自动更新文件继续由 COS 通道提供。 +公开仓库 `Yoan98/Ncurator` 的 Releases 作为安装包下载镜像,由现有平台发布命令通过 GitHub CLI 自动上传与 COS 上线版本相同、已完成正式包回归的文件,不重新构建。GitHub 登录凭证由 CLI 管理,不新增项目密钥配置。Windows 与 macOS 沿用各自独立 Tag,每条 Release 包含对应平台的 EXE 或 DMG、`SHA256SUMS` 和版本说明;自动更新文件继续由 COS 通道提供。 -Release Tag 必须对应实际构建源码。首次公开或补发旧版本时,先核对公开源码快照与原构建代码的对应关系,不把含凭证的私有历史整体推入公开仓库。GitHub 自动附带的 Source code 压缩包不替代安装包,也不替代核对完整对应源码的义务。 +Release 的公开 annotated Tag 必须与候选 bundle 的构建 commit 完全一致;公开源码后从该源码创建新的 Tag 和候选包。自动流程不代建 Tag,不把含凭证的私有历史推入公开仓库,也不处理历史包与公开快照的 commit 映射。GitHub 自动附带的 Source code 压缩包不替代安装包,也不替代核对完整对应源码的义务。 Windows 和 macOS 独立发布,GitHub 的 Latest 标记不能代表两个平台都已更新。README 使用仓库 Releases 列表作为 GitHub 下载入口,不依赖跨平台共用的 `releases/latest/download` 地址。具体上传与恢复操作见[发布流程](./release.md#31-github-releases-镜像)。 仅当有意发布未来海外版时才切 R2(非当前常规循环):`RELEASE_STORAGE_TARGET=r2`,并设置 `R2_ACCESS_KEY_ID`、`R2_SECRET_ACCESS_KEY`、`R2_ACCOUNT_ID`、`R2_BUCKET_NAME`、`R2_PUBLIC_BASE_URL=https://download.ncurator.com`。 -publish 上传候选与 fragment,更新该平台自动更新 metadata 与官网 manifest,验证公开 yml / 清单 / artifacts 的大小、sha512 与 manifest URL(安装包 URL 须以用户可见文件名结尾)。验证成功后标记 `published`,清理本地 `tests/logs/**`,并清理该平台更早 artifacts(含旧 `release-status/`)。失败、prepare-only、test-only 不清理 logs。 +publish 先校验 GitHub 源码身份并准备草稿及附件,再上传 COS 候选与 fragment,更新该平台自动更新 metadata 与官网 manifest,验证公开 yml / 清单 / artifacts 的大小、sha512 与 manifest URL(安装包 URL 须以用户可见文件名结尾);随后公开 GitHub Release,并通过匿名下载核对大小与 SHA-256。两边成功后才标记 `published`,清理本地 `tests/logs/**`,并清理该平台更早 artifacts(含旧 `release-status/`)。失败、prepare-only、test-only 不清理 logs。跨服务发布不具备原子性;中断保留同一候选包,重跑 publish 时检查实际官网与更新通道状态、补齐 GitHub 草稿附件并继续,不重建、不覆盖已有附件,也不把较新通道降级。 ### 本地保留与清理 diff --git a/doc/deploy/release.md b/doc/deploy/release.md index e337efdf..6ac47636 100644 --- a/doc/deploy/release.md +++ b/doc/deploy/release.md @@ -1,6 +1,6 @@ # 桌面端发布流程 -官网和应用内自动更新正式发布使用腾讯云 COS,一次发布完成后不再把同一 Tag 上传到 Cloudflare R2。GitHub Releases 可额外提供同一正式安装包的手动下载镜像,按 3.1 节单独上传;现有 `release:win` / `release:mac` 不会自动创建 GitHub Release。R2 仅供未来海外版显式使用 `RELEASE_STORAGE_TARGET=r2`。当前发布平台为 Windows x64 和 macOS arm64,平台 Tag 分别使用 `windows-vX.Y.Z` 与 `mac-vX.Y.Z`,版本号独立递增。 +官网和应用内自动更新正式发布使用腾讯云 COS,一次发布完成后不再把同一 Tag 上传到 Cloudflare R2。GitHub Releases 同时提供同一正式安装包的下载镜像;`release:win` / `release:mac` 的 `--publish-only` 自动完成两边发布,详见 3.1 节。R2 仅供未来海外版显式使用 `RELEASE_STORAGE_TARGET=r2`。当前发布平台为 Windows x64 和 macOS arm64,平台 Tag 分别使用 `windows-vX.Y.Z` 与 `mac-vX.Y.Z`,版本号独立递增。 ## 1. 发布流程 @@ -12,14 +12,16 @@ 4. 对比当前 Tag 与上一已发布同平台版本,评估改动影响并确定正式包回归范围。 5. 在实际安装的候选包上完成确定范围内的正式包 E2E。 6. 记录构建信息、回归结果和剩余风险。 -7. 发布测试通过的同一份候选包到 COS,验证公开下载和更新通道。 -8. 清理本地临时工作区和过期候选包。 +7. 校验 GitHub 登录、公开仓库权限及 Tag 对应的构建 commit,创建或恢复草稿,上传同一安装包和校验文件。 +8. 将同一候选包发布到 COS,验证官网下载和自动更新通道。 +9. 公开 GitHub Release,匿名下载安装包并校验大小与 SHA-256。 +10. 两边全部验证成功后,标记发布完成并清理本地临时工作区和过期候选包。 候选包测试或公开验证失败时,不得放宽构建、审计、哈希和公开通道校验。 ### 1.1 发布准备与 Tag -必须在干净且与 `origin/main` 完全一致的 `main` 上开始。新版本必须高于该平台当前仍存在的最高 Tag。发版机须具备 live `.env`,包括 `AI_LIVE_KEY_*`;默认 provider 为 `dashscope`。 +必须在干净且与 `origin/main` 完全一致的 `main` 上开始。公开发布前应已完成桌面端仓库迁移,`origin` 指向 `Yoan98/Ncurator`,后续 Tag 与构建均基于公开源码。新版本必须高于该平台当前仍存在的最高 Tag。发版机须具备 live `.env`,包括 `AI_LIVE_KEY_*`;默认 provider 为 `dashscope`。 `release:start` 会依次完成: @@ -50,7 +52,7 @@ pnpm release:abort --tag windows-vX.Y.Z # 或 mac-vX.Y.Z ``` -`release:abort` 拒绝删除已 live 或本地状态为 `published` 的 Tag,同时删除远端和本地 annotated Tag 及该 Tag 的本地候选目录。`main` 上已经产生的 prepare 提交会保留。已上线版本有问题时必须发布更高版本,不能 abort live Tag。 +`release:abort` 拒绝删除已 live 或本地状态为 `published` 的 Tag,同时删除该 Tag 的 GitHub 草稿、远端和本地 annotated Tag 及本地候选目录;若 GitHub Release 已公开则拒绝中止。`main` 上已经产生的 prepare 提交会保留。已上线版本有问题时必须发布更高版本,不能 abort live Tag。 ## 2. 候选包与正式包回归 @@ -65,7 +67,7 @@ prepared → ready → published └→ rejected ``` -正式包 E2E 通过后候选包变为 `ready`;测试失败变为 `rejected`;公开发布并验证成功后变为 `published`。候选包、bundle 和对象存储布局见 [`design.md`](./design.md)。 +正式包 E2E 通过后候选包变为 `ready`;测试失败变为 `rejected`;COS 和 GitHub 均公开发布并验证成功后变为 `published`。候选包、bundle 和对象存储布局见 [`design.md`](./design.md)。 ### 2.2 影响评估 @@ -100,7 +102,7 @@ git diff .. ## 3. 发布与公开验证 -publish 只读取已经测试通过的候选包,不重新构建。默认发布到 COS,验证国内公开下载、更新 metadata、公开 manifest、文件大小、sha512 和 URL 成功后才算完成。对象键、CDN Cache-Control 和公开 URL 见 [`design.md`](./design.md)。 +publish 只读取已经测试通过的候选包,不重新构建。先准备 GitHub 草稿与附件,再发布 COS 并验证国内下载、更新 metadata、公开 manifest、文件大小、sha512 和 URL,最后公开并验证 GitHub Release;两边全部通过才算完成。对象键、CDN Cache-Control 和公开 URL 见 [`design.md`](./design.md)。 真实发布默认锁定。用户已批准本次上线后,在当前终端设置: @@ -114,34 +116,34 @@ export NCURATOR_ALLOW_LIVE_PUBLISH=publish-to-production ### 3.1 GitHub Releases 镜像 -前置条件:桌面端源码已发布至 `Yoan98/Ncurator`,该版本的公开 Tag 与实际构建代码对应;同一安装包已经过正式包回归并在 COS 上线验证。首次公开或补发历史版本时,核对并记录原构建源码与公开快照的对应关系,不直接推送含凭证的私有历史,不把旧安装包挂在无关源码 Tag 下。 - -1. 在发版机使用 `verifyReleaseBundle` 复验原候选 bundle、文件大小与哈希,取 `role: installer` 的 EXE 或 DMG,保持原公开文件名和字节内容。 -2. 准备 `SHA256SUMS`:使用 bundle 中该安装包的 `sha256`,每行格式为 ` <公开安装包文件名>`。准备版本说明文件,写明平台、版本、更新内容、安装方式及对应源码 Tag;有源码快照映射时说明其对应关系。 -3. 在 GitHub 的 Releases 页面新建草稿,选择已核对的 Tag,上传安装包和 `SHA256SUMS`。只上传对应平台的安装包,不上传候选目录、日志、`.env` 或签名材料。 -4. 核对草稿中的平台、Tag、文件名和版本说明后发布。GitHub 自动生成的 Source code 下载项是源码压缩包,普通用户安装应选择 EXE / DMG。 -5. 从公开 Release 下载安装包,核对大小与 SHA-256 和原 bundle 一致,再确认 README 的 Releases 入口可访问该版本。GitHub 失败不改动已上线的 COS 下载和自动更新通道。 - -也可使用已安装并登录、具备目标仓库发布权限的 GitHub CLI 准备草稿。以下命令中的 Tag 和文件路径应替换为核对过的本地文件;`--verify-tag` 防止从默认分支误建 Tag: +发版机安装 [GitHub CLI](https://cli.github.com/),并使用具有 `Yoan98/Ncurator` 写权限的账号完成一次浏览器授权。macOS 可用 `brew install gh`;Windows 在实际发版机安装 CLI 并单独登录。 ```bash -release_tag="mac-vX.Y.Z" -release_installer="/absolute/path/馆长-X.Y.Z-macos-arm64.dmg" -gh release create "$release_tag" "$release_installer" "/absolute/path/SHA256SUMS" \ - --repo Yoan98/Ncurator --verify-tag --draft \ - --title "馆长 X.Y.Z · macOS arm64" \ - --notes-file "/absolute/path/release-notes.md" +gh auth login --hostname github.com --web --git-protocol ssh --skip-ssh-key +gh auth status --hostname github.com ``` -Windows 版本使用 `windows-vX.Y.Z`、对应 EXE 及平台标题。上传中断时检查已有草稿,通过 Releases 页面或 `gh release upload` 补齐缺失附件;不使用 `--clobber` 覆盖已公开安装包。公开版本内容有问题时发布更高版本。 +登录凭证由 CLI 管理,不写入项目 `.env`,无需新增 GitHub Token 配置。每个平台继续使用原有发布命令,GitHub 不另设发布开关或上传步骤。 -操作依据:[GitHub Releases](https://docs.github.com/en/repositories/releasing-projects-on-github/about-releases)、[gh release create](https://cli.github.com/manual/gh_release_create)、[gh release upload](https://cli.github.com/manual/gh_release_upload)。 +`--publish-only` 按以下规则执行: + +1. 复验已测试的候选 bundle,并核对公开仓库的 annotated Tag 指向同一构建 commit。Tag 不存在、源码不一致、登录或权限不足时,在写入 COS 前失败;脚本不会代建或推送 Tag。 +2. 创建 GitHub 草稿,或继续使用同 Tag 的已有草稿。只上传对应平台的 EXE / DMG 和 `SHA256SUMS`;版本说明取 Tag 注释并附源码和官网更新日志入口。保留安装包公开文件名,不上传更新文件、bundle、日志或签名材料。 +3. 校验已有附件的大小和 SHA-256;缺少服务端 digest 的旧附件下载复验。中断后只补齐缺失附件,可清除草稿内上传失败留下的 `starter` 附件;内容不同则失败,不覆盖已有安装包。 +4. 发布并验证 COS。若官网下载与更新通道已经指向该候选包,复验后直接继续 GitHub;若只是上传了文件但尚未切换通道,则继续完成 COS 发布。拒绝把公开通道降级到较旧版本。 +5. 将草稿公开,再通过匿名下载复验附件大小与 SHA-256;已公开且内容相同的 Release 只验证。全部成功后候选包才标记 `published`。 + +Windows 与 macOS 保持独立 Tag 和 Release,不设置跨平台共用的 Latest 标记。GitHub 的 Source code 下载项是源码压缩包,用户安装应选择 EXE / DMG。 + +首次公开应从清理后的公开源码创建 Tag 和候选包,不推送含凭证、内部资料的私有历史。自动发布要求公开 Tag 与候选 commit 完全一致,不处理历史安装包与公开快照的 commit 映射;补发旧版本需另行核对源码对应关系。 + +操作依据:[CLI 登录](https://cli.github.com/manual/gh_auth_login)、[创建 Release](https://cli.github.com/manual/gh_release_create)、[上传附件](https://cli.github.com/manual/gh_release_upload)。 ## 4. 中断与恢复 - Tag 已创建但 prepare 中断:同 Tag 重跑 `--prepare-only`;如果源码已改变且版本尚未上线,先 abort 再用同版本重打。 - 候选包已生成但测试中断:对同一候选包重跑 `--test-only`。 -- 上传或 finalize 中断:对同一候选包恢复 publish,或执行 `release:storage:finalize`。 +- 上传、COS finalize、GitHub 公开或下载验证中断:修复原因后,对同一候选包重跑原平台的 `--publish-only`;按已有附件和公开通道继续,不重新构建、不覆盖已公开附件。COS 已上线但 GitHub 失败时保留候选包和测试证据,不能宣称整次发布完成;`release:storage:finalize` 只恢复 COS,不完成 GitHub。 - 未上线整轮作废:abort 后用同一版本重打,不要另起版本号。 - 已发布版本有问题:发布更高版本;回退下载通道不会让已安装客户端自动降级。 - 同一 Tag 的对象已存在但哈希不同:禁止覆盖;未上线时执行 abort 后用同一版本重打。 diff --git a/scripts/release/abort.mjs b/scripts/release/abort.mjs index f7e41285..4aff5b02 100644 --- a/scripts/release/abort.mjs +++ b/scripts/release/abort.mjs @@ -6,6 +6,7 @@ import { promisify } from 'node:util' import { resolveCandidateDirectory } from './candidate.mjs' import { findLiveChannelsForTag } from './shared/live-tag.mjs' import { parseReleaseTag } from './tag.mjs' +import { createGitHubReleasePublisher } from './github.mjs' const execFileAsync = promisify(execFile) const ROOT = process.cwd() @@ -98,6 +99,8 @@ if (!remoteExists && !localExists) { throw new Error(`tag not found locally or on origin: ${releaseTag}`) } +await createGitHubReleasePublisher({ root: ROOT }).deleteDraft(releaseTag) + if (remoteExists) { await execFileAsync('git', ['push', 'origin', `:refs/tags/${releaseTag}`], { cwd: ROOT }) process.stdout.write(`[release] deleted origin tag ${releaseTag}\n`) diff --git a/scripts/release/candidate.mjs b/scripts/release/candidate.mjs index ed1b4949..c0b74fb7 100644 --- a/scripts/release/candidate.mjs +++ b/scripts/release/candidate.mjs @@ -152,14 +152,22 @@ export const recordCandidateE2E = async ({ return { ...candidate, state } } -export const assertCandidatePublishable = async ({ tag, platform, commit }) => { +export const assertCandidatePublishable = async ({ + tag, + platform, + commit, + allowPublished = false +}) => { const candidate = await readReleaseCandidate({ tag, platform }) if (candidate.state.commit !== commit) { throw new Error( `candidate commit ${candidate.state.commit} does not match ${tag} commit ${commit}` ) } - if (candidate.state.status !== 'ready') { + if ( + candidate.state.status !== 'ready' && + !(allowPublished && candidate.state.status === 'published') + ) { throw new Error( `candidate ${tag} ${platform} is ${candidate.state.status}; prepare or test it before publishing` ) diff --git a/scripts/release/github.mjs b/scripts/release/github.mjs new file mode 100644 index 00000000..a38df6c6 --- /dev/null +++ b/scripts/release/github.mjs @@ -0,0 +1,234 @@ +import { execFile } from 'node:child_process' +import { createHash } from 'node:crypto' +import { copyFile, mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises' +import os from 'node:os' +import path from 'node:path' +import { promisify } from 'node:util' +import { resolveReleaseAssetPath } from './r2/fragment-contract.mjs' +import { assertLivePublishAllowed } from './storage/config.mjs' +import { parseReleaseTag } from './tag.mjs' +import { fetchWithRetry } from './shared/http.mjs' + +export const GITHUB_RELEASE_REPOSITORY = 'Yoan98/Ncurator' +const REPOSITORY_PATH = `repos/${GITHUB_RELEASE_REPOSITORY}` +const REPOSITORY_URL = `https://github.com/${GITHUB_RELEASE_REPOSITORY}` +const execFileAsync = promisify(execFile) +const sha256 = (content) => createHash('sha256').update(content).digest('hex') + +export const createGitHubReleasePublisher = ({ + root = process.cwd(), + execute = execFileAsync, + fetchImpl = fetchWithRetry +} = {}) => { + const gh = async (args) => { + try { + return await execute('gh', args, { + cwd: root, + timeout: 20 * 60_000, + maxBuffer: 10 * 1024 * 1024, + env: { ...process.env, GH_PROMPT_DISABLED: '1' } + }) + } catch (error) { + if (error.code === 'ENOENT') { + throw new Error( + 'GitHub CLI is missing; install gh and run gh auth login --hostname github.com' + ) + } + throw error + } + } + const api = async (endpoint) => + JSON.parse((await gh(['api', '--hostname', 'github.com', '--method', 'GET', endpoint])).stdout) + const releaseArgs = (tag) => [tag, '--repo', `github.com/${GITHUB_RELEASE_REPOSITORY}`] + + const readRelease = async (tag) => { + try { + // CLI resolves draft releases too; REST's /releases/tags endpoint only finds published ones. + const { databaseId } = JSON.parse( + (await gh(['release', 'view', ...releaseArgs(tag), '--json', 'databaseId'])).stdout + ) + return await api(`${REPOSITORY_PATH}/releases/${databaseId}`) + } catch (error) { + if (/^release not found\s*$/mu.test(String(error.stderr || error.message))) return undefined + throw error + } + } + + const verifySource = async (bundle) => { + const repository = await api(REPOSITORY_PATH) + if (repository.private || !repository.permissions?.push) { + throw new Error( + `GitHub publication requires write access to public ${GITHUB_RELEASE_REPOSITORY}` + ) + } + const reference = await api(`${REPOSITORY_PATH}/git/ref/tags/${bundle.tag}`) + if (reference.object?.type !== 'tag') { + throw new Error(`GitHub ${bundle.tag} must be an annotated tag`) + } + const tag = await api(`${REPOSITORY_PATH}/git/tags/${reference.object.sha}`) + if (tag.object?.type !== 'commit' || tag.object.sha !== bundle.commit) { + throw new Error(`GitHub ${bundle.tag} does not match candidate commit ${bundle.commit}`) + } + return tag.message.trim() + } + + const verifyAssets = async (release, files) => { + if (!release || release.prerelease || release.assets.length !== files.length) { + throw new Error('GitHub Release must contain only the installer and SHA256SUMS') + } + for (const file of files) { + const asset = release.assets.find((entry) => entry.name === file.name) + if (!asset || asset.state !== 'uploaded' || asset.size !== file.size) { + throw new Error(`GitHub Release asset is missing or incomplete: ${file.name}`) + } + if (asset.digest) { + if (asset.digest !== `sha256:${file.sha256}`) { + throw new Error(`GitHub Release asset hash differs: ${file.name}`) + } + } else { + // Older GitHub assets may have no digest. Verify their bytes before reusing them. + const directory = await mkdtemp(path.join(os.tmpdir(), 'ncurator-github-verify-')) + try { + await gh([ + 'release', + 'download', + ...releaseArgs(release.tag_name), + '--pattern', + file.name, + '--dir', + directory + ]) + const content = await readFile(path.join(directory, file.name)) + if (content.length !== file.size || sha256(content) !== file.sha256) { + throw new Error(`GitHub Release asset hash differs: ${file.name}`) + } + } finally { + await rm(directory, { recursive: true, force: true }) + } + } + } + } + + return { + async prepare(candidate) { + assertLivePublishAllowed() + if (!['ready', 'published'].includes(candidate.state.status)) { + throw new Error('GitHub publication requires a tested release candidate') + } + const { bundle, directory } = candidate.verified + const annotation = await verifySource(bundle) + const installer = bundle.files.find((file) => file.role === 'installer') + const staging = path.join(candidate.directory, 'github') + await mkdir(staging, { recursive: true }) + const installerPath = await resolveReleaseAssetPath(directory, installer) + const uploadPath = + path.basename(installerPath) === installer.name + ? installerPath + : path.join(staging, installer.name) + if (uploadPath !== installerPath) await copyFile(installerPath, uploadPath) + const sums = `${installer.sha256} ${installer.name}\n` + const sumsPath = path.join(staging, 'SHA256SUMS') + await writeFile(sumsPath, sums, 'utf8') + const files = [ + { ...installer, path: uploadPath }, + { name: 'SHA256SUMS', path: sumsPath, size: Buffer.byteLength(sums), sha256: sha256(sums) } + ] + const { family, version } = parseReleaseTag(bundle.tag) + const platform = family === 'windows' ? 'Windows x64' : 'macOS arm64' + const notesPath = path.join(staging, 'release-notes.md') + await writeFile( + notesPath, + [ + annotation, + '', + `平台:${platform}。下载下方的 ${family === 'windows' ? 'EXE' : 'DMG'} 安装包。`, + '安装包与官网下载版本一致;SHA256SUMS 提供 SHA-256 校验值。', + '', + `[对应源码](${REPOSITORY_URL}/tree/${bundle.tag}) · [官网更新日志](https://www.guanzhangai.cn/zh/changelog)`, + '' + ].join('\n'), + 'utf8' + ) + + let release = await readRelease(bundle.tag) + if (!release) { + await gh([ + 'release', + 'create', + ...releaseArgs(bundle.tag), + '--verify-tag', + '--draft', + '--latest=false', + '--title', + `馆长 ${version} · ${platform}`, + '--notes-file', + notesPath + ]) + release = await readRelease(bundle.tag) + } + if (!release) throw new Error(`GitHub Release draft is missing: ${bundle.tag}`) + if (release.draft) { + const expectedNames = new Set(files.map((file) => file.name)) + if (release.prerelease || release.assets.some((asset) => !expectedNames.has(asset.name))) { + throw new Error('GitHub draft contains unexpected assets or is marked prerelease') + } + for (const file of files) { + const existing = release.assets.find((asset) => asset.name === file.name) + if (existing?.state === 'starter') { + await gh(['release', 'delete-asset', ...releaseArgs(bundle.tag), file.name, '--yes']) + } else if (existing) { + continue + } + await gh(['release', 'upload', ...releaseArgs(bundle.tag), file.path]) + } + release = await readRelease(bundle.tag) + } + await verifyAssets(release, files) + process.stdout.write(`[release] GitHub assets verified for ${bundle.tag}\n`) + return { bundle, files, releaseId: release.id } + }, + + async publish(prepared) { + assertLivePublishAllowed() + const { bundle, files, releaseId } = prepared + await verifySource(bundle) + let release = await readRelease(bundle.tag) + if (release?.id !== releaseId) throw new Error('GitHub Release changed during publication') + await verifyAssets(release, files) + if (release.draft) { + await gh(['release', 'edit', ...releaseArgs(bundle.tag), '--draft=false', '--latest=false']) + release = await readRelease(bundle.tag) + } + if (!release || release.draft) throw new Error('GitHub Release is not public') + await verifyAssets(release, files) + // Anonymous downloads verify that users can fetch the same bytes, independently of CLI auth. + for (const file of files) { + const asset = release.assets.find((entry) => entry.name === file.name) + const response = await fetchImpl(asset.browser_download_url, { + cache: 'no-store', + signal: AbortSignal.timeout(20 * 60_000) + }) + if (!response.ok || !response.body) { + throw new Error(`GitHub public download failed: ${file.name} (HTTP ${response.status})`) + } + const hash = createHash('sha256') + let size = 0 + for await (const chunk of response.body) { + size += chunk.length + hash.update(chunk) + } + if (size !== file.size || hash.digest('hex') !== file.sha256) { + throw new Error(`GitHub public download hash differs: ${file.name}`) + } + } + process.stdout.write(`[release] GitHub ${bundle.tag} is public: ${release.html_url}\n`) + }, + + async deleteDraft(tag) { + const release = await readRelease(tag) + if (!release) return + if (!release.draft) throw new Error(`GitHub ${tag} is published; refuse to abort`) + await gh(['release', 'delete', ...releaseArgs(tag), '--yes']) + } + } +} diff --git a/scripts/release/macos.mjs b/scripts/release/macos.mjs index bc6e0c29..b8672fa1 100644 --- a/scripts/release/macos.mjs +++ b/scripts/release/macos.mjs @@ -23,6 +23,12 @@ import { validateReleaseArgs } from './regression.mjs' import { assertLivePublishAllowed, resolveReleaseStorageConfig } from './storage/config.mjs' +import { createGitHubReleasePublisher } from './github.mjs' +import { + isReleaseChannelCurrent, + verifyPublicReleaseFragment, + verifyReleaseStorageFragment +} from './shared/live-tag.mjs' const execFileAsync = promisify(execFile) const ROOT = process.cwd() @@ -320,18 +326,37 @@ if (mode === 'test') { const candidate = await assertCandidatePublishable({ tag: releaseTag, platform: MACOS_PLATFORM, - commit: taggedCommit + commit: taggedCommit, + allowPublished: true }) -await run(process.execPath, [ - path.join(ROOT, 'scripts', 'release', 'r2', 'upload-fragment.mjs'), - candidate.verified.directory, - candidate.verified.fragmentPath -]) -await run(process.execPath, [ - path.join(ROOT, 'scripts', 'release', 'r2', 'finalize-macos.mjs'), - '--tag', - releaseTag -]) +const storage = resolveReleaseStorageConfig() +const github = createGitHubReleasePublisher({ root: ROOT }) +const channelCurrent = await isReleaseChannelCurrent( + candidate.verified.fragment, + storage.publicBaseUrl +) +if (!channelCurrent && candidate.state.status === 'published') { + throw new Error(`published ${releaseTag} is missing from ${storage.label}; refuse to republish`) +} +const githubRelease = await github.prepare(candidate) +if (!channelCurrent) { + await run(process.execPath, [ + path.join(ROOT, 'scripts', 'release', 'r2', 'upload-fragment.mjs'), + candidate.verified.directory, + candidate.verified.fragmentPath + ]) + await run(process.execPath, [ + path.join(ROOT, 'scripts', 'release', 'r2', 'finalize-macos.mjs'), + '--tag', + releaseTag + ]) +} +await verifyPublicReleaseFragment(candidate.verified.fragment, storage.publicBaseUrl) +await verifyReleaseStorageFragment(candidate.verified.fragment, process.env.NCURATOR_AWS_CLI) +if (!(await isReleaseChannelCurrent(candidate.verified.fragment, storage.publicBaseUrl))) { + throw new Error('public macOS release channel does not match the candidate') +} +await github.publish(githubRelease) await markCandidatePublished({ tag: releaseTag, platform: MACOS_PLATFORM }) try { await cleanupLocalTestLogs(ROOT) @@ -351,4 +376,4 @@ try { const message = error instanceof Error ? error.message : String(error) process.stderr.write(`[release] local candidate cleanup warning: ${message}\n`) } -process.stdout.write(`[release] macOS ${releaseTag} published from retained candidates\n`) +process.stdout.write(`[release] macOS ${releaseTag} published to ${storage.label} and GitHub\n`) diff --git a/scripts/release/shared/live-tag.mjs b/scripts/release/shared/live-tag.mjs index a071bc66..1b6316ab 100644 --- a/scripts/release/shared/live-tag.mjs +++ b/scripts/release/shared/live-tag.mjs @@ -1,16 +1,81 @@ +import { execFile } from 'node:child_process' +import { promisify } from 'node:util' import { DOMESTIC_DOWNLOAD_BASE_URL, - GLOBAL_DOWNLOAD_BASE_URL + GLOBAL_DOWNLOAD_BASE_URL, + resolveReleaseStorageConfig } from '../storage/config.mjs' import { fetchWithRetry } from './http.mjs' -import { parseReleaseTag } from '../tag.mjs' +import { compareReleaseVersions, parseReleaseTag } from '../tag.mjs' +import { artifactKey } from '../r2/storage-layout.mjs' +import YAML from 'yaml' + +const execFileAsync = promisify(execFile) const LATEST_MANIFEST = { mac: 'releases/latest/macos.json', windows: 'releases/latest/windows.json' } -export const RELEASE_PUBLIC_BASE_URLS = [DOMESTIC_DOWNLOAD_BASE_URL, GLOBAL_DOWNLOAD_BASE_URL] +export const RELEASE_PUBLIC_BASE_URLS = [ + DOMESTIC_DOWNLOAD_BASE_URL, + GLOBAL_DOWNLOAD_BASE_URL, + resolveReleaseStorageConfig +] + +export const isReleaseChannelCurrent = async (fragment, base, fetchImpl = fetchWithRetry) => { + const { family, version } = parseReleaseTag(fragment.tag, fragment.platform) + const metadataPath = + family === 'mac' ? 'updates/latest/latest-mac.yml' : 'updates/latest/latest.yml' + const responses = await Promise.all( + [LATEST_MANIFEST[family], metadataPath].map((key) => + fetchImpl(`${base}/${key}?check=${Date.now()}`, { cache: 'no-store' }) + ) + ) + for (const response of responses) { + if (!response.ok && response.status !== 404) { + throw new Error(`public release channel lookup failed: HTTP ${response.status}`) + } + } + const manifest = responses[0].ok ? await responses[0].json() : undefined + const metadata = responses[1].ok ? YAML.parse(await responses[1].text()) : undefined + for (const current of [manifest, metadata]) { + if (!current) continue + if (!/^\d+\.\d+\.\d+$/u.test(current.version)) { + throw new Error('public release channel version is invalid') + } + if (compareReleaseVersions(current.version, version) > 0) { + throw new Error( + `public channel already contains newer ${current.version}; refuse to downgrade` + ) + } + } + if (manifest?.version === version && manifest.tag !== fragment.tag) { + throw new Error(`public channel tag differs from ${fragment.tag}`) + } + const installer = fragment.files.find((file) => file.role === 'installer') + const download = manifest?.downloads?.[family === 'mac' ? 'arm64' : 'x64'] + const expectedUpdates = fragment.files.filter((file) => + family === 'mac' ? file.name.endsWith('.zip') : file.role === 'installer' + ) + return ( + manifest?.tag === fragment.tag && + metadata?.version === version && + download?.name === installer.name && + download.size === installer.size && + download.sha512 === installer.sha512 && + download.url === `${base}/${artifactKey(fragment, installer.name)}` && + expectedUpdates.every((file) => + metadata.files?.some( + (entry) => + decodeURI(new URL(entry.url, `${base}/${metadataPath}`).href) === + `${base}/${artifactKey(fragment, file.name)}` && + entry.sha512 === file.sha512 && + entry.size === file.size + ) + ) + ) +} export const findLiveChannelsForTag = async (releaseTag, fetchImpl = fetchWithRetry) => { const { family } = parseReleaseTag(releaseTag) @@ -28,3 +93,47 @@ export const findLiveChannelsForTag = async (releaseTag, fetchImpl = fetchWithRe return live } + +export const verifyPublicReleaseFragment = async ( + fragment, + publicBaseUrl = DOMESTIC_DOWNLOAD_BASE_URL +) => { + for (const file of fragment.files) { + const response = await fetchWithRetry(`${publicBaseUrl}/${artifactKey(fragment, file.name)}`, { + method: 'HEAD', + cache: 'no-store' + }) + if (!response.ok || Number(response.headers.get('content-length')) !== file.size) { + throw new Error(`remote release asset verification failed: ${file.name}`) + } + } +} + +export const verifyReleaseStorageFragment = async (fragment, awsExecutable) => { + const config = resolveReleaseStorageConfig() + if (!awsExecutable) throw new Error(`${config.label} verification configuration is incomplete`) + const { createAwsCliRuntime } = await import('../storage/client.mjs') + const runtime = await createAwsCliRuntime(config) + for (const file of fragment.files) { + const key = artifactKey(fragment, file.name) + const result = await execFileAsync( + awsExecutable, + [ + 's3api', + 'head-object', + '--bucket', + config.bucket, + '--key', + key, + '--output', + 'json', + ...runtime.extraArgs + ], + { env: runtime.env } + ) + const remote = JSON.parse(result.stdout) + if (Number(remote.ContentLength) !== file.size || remote.Metadata?.sha512 !== file.sha512) { + throw new Error(`${config.label} release asset does not match fragment: ${key}`) + } + } +} diff --git a/scripts/release/shared/windows.mjs b/scripts/release/shared/windows.mjs index dadb1f1e..b83889b2 100644 --- a/scripts/release/shared/windows.mjs +++ b/scripts/release/shared/windows.mjs @@ -5,14 +5,9 @@ import path from 'node:path' import process from 'node:process' import { promisify } from 'node:util' import { loadReleaseEnvironment } from './environment.mjs' -import { assertReleaseFragment } from '../r2/fragment-contract.mjs' -import { fetchWithRetry } from './http.mjs' -import { artifactKey, fragmentKey } from '../r2/storage-layout.mjs' -import { DOMESTIC_DOWNLOAD_BASE_URL, resolveReleaseStorageConfig } from '../storage/config.mjs' +import { resolveReleaseStorageConfig } from '../storage/config.mjs' const execFileAsync = promisify(execFile) -const WINDOWS_PLATFORM = 'windows-x64' -const PUBLIC_RELEASE_BASE_URL = DOMESTIC_DOWNLOAD_BASE_URL const commandResult = async (command, args, options = {}) => { try { @@ -131,6 +126,11 @@ export const inspectLocalWindowsRelease = async (root, { requirePublish = true } const awsExecutable = requirePublish ? await resolveAwsExecutable() : '' if (requirePublish && !awsExecutable) issues.push('AWS CLI is not available') + if (requirePublish && !(await commandResult('gh', ['--version'])).ok) { + issues.push( + 'GitHub CLI is not available; install gh and run gh auth login --hostname github.com' + ) + } for (const [command, label] of [ ['git', 'Git'], @@ -155,75 +155,3 @@ export const inspectLocalWindowsRelease = async (root, { requirePublish = true } python } } - -export const readPublicReleaseFragment = async ( - releaseTag, - platform, - publicBaseUrl = PUBLIC_RELEASE_BASE_URL -) => { - const response = await fetch( - `${publicBaseUrl}/${fragmentKey(releaseTag, platform)}?check=${Date.now()}`, - { cache: 'no-store' } - ) - if (response.status === 404) return undefined - if (!response.ok) { - throw new Error(`release status lookup failed for ${platform}: HTTP ${response.status}`) - } - const fragment = await response.json() - assertReleaseFragment(fragment, { - tag: releaseTag, - platform, - label: `remote release fragment ${platform}` - }) - return fragment -} - -export const verifyPublicReleaseFragment = async ( - fragment, - publicBaseUrl = PUBLIC_RELEASE_BASE_URL -) => { - for (const file of fragment.files) { - const response = await fetchWithRetry( - `${publicBaseUrl}/${artifactKey(fragment, file.name)}`, - { - method: 'HEAD', - cache: 'no-store' - } - ) - if (!response.ok || Number(response.headers.get('content-length')) !== file.size) { - throw new Error(`remote release asset verification failed: ${file.name}`) - } - } -} - -export const verifyReleaseStorageFragment = async (fragment, awsExecutable) => { - const config = resolveReleaseStorageConfig() - if (!awsExecutable) throw new Error(`${config.label} verification configuration is incomplete`) - const { createAwsCliRuntime } = await import('../storage/client.mjs') - const runtime = await createAwsCliRuntime(config) - for (const file of fragment.files) { - const key = artifactKey(fragment, file.name) - const result = await commandResult( - awsExecutable, - [ - 's3api', - 'head-object', - '--bucket', - config.bucket, - '--key', - key, - '--output', - 'json', - ...runtime.extraArgs - ], - { env: runtime.env } - ) - if (!result.ok) throw new Error(`${config.label} release asset lookup failed: ${key}`) - const remote = JSON.parse(result.stdout) - if (Number(remote.ContentLength) !== file.size || remote.Metadata?.sha512 !== file.sha512) { - throw new Error(`${config.label} release asset does not match fragment: ${key}`) - } - } -} - -export { PUBLIC_RELEASE_BASE_URL, WINDOWS_PLATFORM } diff --git a/scripts/release/windows.mjs b/scripts/release/windows.mjs index 1b80127b..800fde87 100644 --- a/scripts/release/windows.mjs +++ b/scripts/release/windows.mjs @@ -12,12 +12,7 @@ import { readReleaseCandidate, recordCandidateE2E } from './candidate.mjs' -import { - inspectLocalWindowsRelease, - readPublicReleaseFragment, - verifyPublicReleaseFragment, - verifyReleaseStorageFragment -} from './shared/windows.mjs' +import { inspectLocalWindowsRelease } from './shared/windows.mjs' import { assertLivePublishAllowed, resolveReleaseStorageConfig } from './storage/config.mjs' import { prepareReleaseWorkspace } from './workspace.mjs' import { parseReleaseTag } from './tag.mjs' @@ -29,6 +24,12 @@ import { import { cleanupLocalTestLogs } from './shared/test-logs.mjs' import { createReleaseStageDeadline } from './shared/stage-deadline.mjs' import { loadReleaseEnvironment } from './shared/environment.mjs' +import { createGitHubReleasePublisher } from './github.mjs' +import { + isReleaseChannelCurrent, + verifyPublicReleaseFragment, + verifyReleaseStorageFragment +} from './shared/live-tag.mjs' import { cpSync, existsSync } from 'node:fs' const execFileAsync = promisify(execFile) @@ -307,37 +308,41 @@ if (mode === 'test') { const storage = resolveReleaseStorageConfig() const publicBaseUrl = storage.publicBaseUrl -const existing = await readPublicReleaseFragment(releaseTag, 'windows-x64', publicBaseUrl) -if (existing) { - await verifyPublicReleaseFragment(existing, publicBaseUrl) - await verifyReleaseStorageFragment(existing, inspection.awsExecutable) - await cleanupLocalTestLogsSafely() - await cleanupLocalCandidates() - process.stdout.write(`[release] Windows ${releaseTag} is already complete on ${storage.label}\n`) - process.exit(0) -} const candidate = await assertCandidatePublishable({ tag: releaseTag, platform: 'windows-x64', - commit: taggedCommit + commit: taggedCommit, + allowPublished: true }) -await run( - process.execPath, - [ - path.join(ROOT, 'scripts', 'release', 'r2', 'upload-fragment.mjs'), - candidate.verified.directory, - candidate.verified.fragmentPath - ], - { cwd: ROOT, env: process.env } -) -await run( - process.execPath, - [path.join(ROOT, 'scripts', 'release', 'r2', 'finalize-windows.mjs'), '--tag', releaseTag], - { cwd: ROOT, env: process.env } -) +const github = createGitHubReleasePublisher({ root: ROOT }) +const channelCurrent = await isReleaseChannelCurrent(candidate.verified.fragment, publicBaseUrl) +if (!channelCurrent && candidate.state.status === 'published') { + throw new Error(`published ${releaseTag} is missing from ${storage.label}; refuse to republish`) +} +const githubRelease = await github.prepare(candidate) +if (!channelCurrent) { + await run( + process.execPath, + [ + path.join(ROOT, 'scripts', 'release', 'r2', 'upload-fragment.mjs'), + candidate.verified.directory, + candidate.verified.fragmentPath + ], + { cwd: ROOT, env: process.env } + ) + await run( + process.execPath, + [path.join(ROOT, 'scripts', 'release', 'r2', 'finalize-windows.mjs'), '--tag', releaseTag], + { cwd: ROOT, env: process.env } + ) +} await verifyPublicReleaseFragment(candidate.verified.fragment, publicBaseUrl) await verifyReleaseStorageFragment(candidate.verified.fragment, inspection.awsExecutable) +if (!(await isReleaseChannelCurrent(candidate.verified.fragment, publicBaseUrl))) { + throw new Error('public Windows release channel does not match the candidate') +} +await github.publish(githubRelease) await markCandidatePublished({ tag: releaseTag, platform: 'windows-x64' }) await cleanupLocalTestLogsSafely() await cleanupLocalCandidates() -process.stdout.write(`[release] Windows ${releaseTag} published from ${candidate.bundlePath}\n`) +process.stdout.write(`[release] Windows ${releaseTag} published to ${storage.label} and GitHub\n`) diff --git a/tests/case/unit/release/app-update-publish-config.test.ts b/tests/case/unit/release/app-update-publish-config.test.ts index 086c7eb2..3486e3ec 100644 --- a/tests/case/unit/release/app-update-publish-config.test.ts +++ b/tests/case/unit/release/app-update-publish-config.test.ts @@ -140,7 +140,6 @@ describe('app update publish config', () => { macFinalize, tagContract, storageConfig, - windowsShared, uploadFragment ] = await Promise.all([ readFile(path.resolve('scripts/release/windows.mjs'), 'utf8'), @@ -149,7 +148,6 @@ describe('app update publish config', () => { readFile(path.resolve('scripts/release/r2/finalize-macos.mjs'), 'utf8'), readFile(path.resolve('scripts/release/tag.mjs'), 'utf8'), readFile(path.resolve('scripts/release/storage/config.mjs'), 'utf8'), - readFile(path.resolve('scripts/release/shared/windows.mjs'), 'utf8'), readFile(path.resolve('scripts/release/r2/upload-fragment.mjs'), 'utf8') ]) @@ -177,9 +175,6 @@ describe('app update publish config', () => { expect(macFinalize).not.toContain('release-status/macos/') expect(uploadFragment).toContain('fragmentKey') expect(uploadFragment).toContain("'no-store'") - expect(windowsShared).toContain('fragmentKey') - expect(windowsShared).toContain('artifactKey(fragment, file.name)') - expect(windowsShared).not.toContain('file.storageName') expect(tagContract).toContain('/^(mac|windows)-v') expect(windowsRelease).toContain('assertLivePublishAllowed') expect(macRelease).toContain('assertLivePublishAllowed') diff --git a/tests/case/unit/release/artifact-bundle.test.ts b/tests/case/unit/release/artifact-bundle.test.ts index 5651d4ee..64779075 100644 --- a/tests/case/unit/release/artifact-bundle.test.ts +++ b/tests/case/unit/release/artifact-bundle.test.ts @@ -1,7 +1,9 @@ import fs from 'node:fs/promises' import os from 'node:os' import path from 'node:path' -import { afterEach, describe, expect, it } from 'vitest' +import { afterEach, beforeEach, describe, expect, it } from 'vitest' +import { createHash } from 'node:crypto' +import { createGitHubReleasePublisher } from '../../../../scripts/release/github.mjs' import { createReleaseBundle, verifyReleaseBundle @@ -98,3 +100,196 @@ describe('release artifact bundle', () => { await expect(verifyReleaseBundle(bundlePath)).rejects.toThrow('release asset size changed') }) }) + +type GitHubAsset = { + name: string + state: string + size: number + digest: string + browser_download_url: string +} +type GitHubRelease = { + id: number + tag_name: string + draft: boolean + prerelease: boolean + assets: GitHubAsset[] + html_url: string +} + +const githubHarness = () => { + const state = { + commit: '0123456789012345678901234567890123456789', + release: undefined as GitHubRelease | undefined, + failUpload: '', + failPublication: false, + writes: [] as string[], + bytes: new Map() + } + const execute = async (command: string, args: string[]) => { + expect(command).toBe('gh') + if (args[0] === 'release' && args[1] === 'view') { + if (!state.release) + throw Object.assign(new Error('not found'), { stderr: 'release not found' }) + return { stdout: JSON.stringify({ databaseId: state.release.id }) } + } + if (args[0] === 'api') { + const endpoint = args.at(-1) + if (endpoint === 'repos/Yoan98/Ncurator') { + return { stdout: JSON.stringify({ private: false, permissions: { push: true } }) } + } + if (endpoint?.includes('/git/ref/')) { + return { stdout: JSON.stringify({ object: { type: 'tag', sha: 'annotated-tag' } }) } + } + if (endpoint?.includes('/git/tags/')) { + return { + stdout: JSON.stringify({ + object: { type: 'commit', sha: state.commit }, + message: '修复文件读取' + }) + } + } + if (!state.release) throw Object.assign(new Error('not found'), { stderr: '(HTTP 404)' }) + return { stdout: JSON.stringify(state.release) } + } + state.writes.push(args[1]) + if (args[1] === 'create') { + expect(args).toContain('--verify-tag') + expect(args).toContain('--draft') + state.release = { + id: 7, + tag_name: args[2], + draft: true, + prerelease: false, + assets: [], + html_url: `https://github.com/Yoan98/Ncurator/releases/tag/${args[2]}` + } + } else if (args[1] === 'upload') { + expect(args).not.toContain('--clobber') + const filePath = args.at(-1)! + const name = path.basename(filePath) + if (state.failUpload === name) throw new Error('upload interrupted') + const content = await fs.readFile(filePath) + const url = `https://github.com/Yoan98/Ncurator/releases/download/${args[2]}/${encodeURIComponent(name)}` + state.bytes.set(url, content) + state.release!.assets.push({ + name, + size: content.length, + state: 'uploaded', + digest: `sha256:${createHash('sha256').update(content).digest('hex')}`, + browser_download_url: url + }) + } else if (args[1] === 'edit') { + if (state.failPublication) throw new Error('publication interrupted') + state.release!.draft = false + } else if (args[1] === 'delete') { + state.release = undefined + } else { + throw new Error(`unexpected command: ${args.join(' ')}`) + } + return { stdout: '' } + } + const publisher = createGitHubReleasePublisher({ + execute, + fetchImpl: async (url: string) => { + const content = state.bytes.get(url) + return new Response(content ? new Uint8Array(content) : null, { status: content ? 200 : 404 }) + } + }) + return { state, publisher } +} + +const githubCandidate = async () => { + const directory = await createFixture() + const bundlePath = path.join(directory, 'release-bundle-windows-x64.json') + await createReleaseBundle({ + fragmentPath: path.join(directory, 'windows-x64.json'), + outputPath: bundlePath, + commit: '0123456789012345678901234567890123456789' + }) + return { directory, state: { status: 'ready' }, verified: await verifyReleaseBundle(bundlePath) } +} + +describe('GitHub release publication', () => { + const previousConfirmation = process.env.NCURATOR_ALLOW_LIVE_PUBLISH + beforeEach(() => { + process.env.NCURATOR_ALLOW_LIVE_PUBLISH = 'publish-to-production' + }) + afterEach(() => { + if (previousConfirmation === undefined) delete process.env.NCURATOR_ALLOW_LIVE_PUBLISH + else process.env.NCURATOR_ALLOW_LIVE_PUBLISH = previousConfirmation + }) + + it('uploads only the tested installer and checksum, then publishes verified public bytes', async () => { + const { state, publisher } = githubHarness() + const prepared = await publisher.prepare(await githubCandidate()) + expect(state.release?.draft).toBe(true) + expect(state.release?.assets.map((asset) => asset.name)).toEqual([ + '馆长-0.1.21-Windows-x64.exe', + 'SHA256SUMS' + ]) + await publisher.publish(prepared) + expect(state.release?.draft).toBe(false) + const writes = [...state.writes] + await publisher.publish(await publisher.prepare(await githubCandidate())) + expect(state.writes).toEqual(writes) + }) + + it('stops before any write if source identity or the live gate is invalid', async () => { + const { state, publisher } = githubHarness() + state.commit = 'f'.repeat(40) + await expect(publisher.prepare(await githubCandidate())).rejects.toThrow('does not match') + expect(state.writes).toEqual([]) + delete process.env.NCURATOR_ALLOW_LIVE_PUBLISH + await expect(publisher.prepare(await githubCandidate())).rejects.toThrow( + 'live publish is locked' + ) + expect(state.writes).toEqual([]) + }) + + it('resumes an interrupted upload without overwriting the existing installer', async () => { + const { state, publisher } = githubHarness() + const candidate = await githubCandidate() + state.failUpload = 'SHA256SUMS' + await expect(publisher.prepare(candidate)).rejects.toThrow('upload interrupted') + expect(state.release?.assets).toHaveLength(1) + state.failUpload = '' + await publisher.publish(await publisher.prepare(candidate)) + expect(state.release?.assets).toHaveLength(2) + expect(state.writes.filter((write) => write === 'create')).toHaveLength(1) + }) + + it('resumes publication after the website stage without creating a second release', async () => { + const { state, publisher } = githubHarness() + const candidate = await githubCandidate() + const prepared = await publisher.prepare(candidate) + state.failPublication = true + await expect(publisher.publish(prepared)).rejects.toThrow('publication interrupted') + state.failPublication = false + await publisher.publish(await publisher.prepare(candidate)) + expect(state.release?.draft).toBe(false) + expect(state.writes.filter((write) => write === 'upload')).toHaveLength(2) + }) + + it('refuses mismatched existing assets and detects corrupt public downloads', async () => { + const { state, publisher } = githubHarness() + const candidate = await githubCandidate() + const prepared = await publisher.prepare(candidate) + state.release!.assets[0].digest = `sha256:${'f'.repeat(64)}` + await expect(publisher.prepare(candidate)).rejects.toThrow('asset hash differs') + state.release!.assets[0].digest = `sha256:${candidate.verified.bundle.files[0].sha256}` + state.bytes.set(state.release!.assets[0].browser_download_url, Buffer.from('corrupted')) + await expect(publisher.publish(prepared)).rejects.toThrow('public download hash differs') + }) + + it('deletes unpublished drafts on abort and protects published releases', async () => { + const { publisher } = githubHarness() + const candidate = await githubCandidate() + await publisher.prepare(candidate) + await publisher.deleteDraft(candidate.verified.bundle.tag) + await publisher.publish(await publisher.prepare(candidate)) + await expect(publisher.deleteDraft(candidate.verified.bundle.tag)).rejects.toThrow( + 'refuse to abort' + ) + }) +}) diff --git a/tests/case/unit/release/candidate.test.ts b/tests/case/unit/release/candidate.test.ts index d44b4bc6..2576a7ce 100644 --- a/tests/case/unit/release/candidate.test.ts +++ b/tests/case/unit/release/candidate.test.ts @@ -8,6 +8,7 @@ import { assertCandidatePublishable, cleanupSupersededReleaseCandidates, materializeReleaseCandidate, + markCandidatePublished, readReleaseCandidate, recordCandidateE2E, resolveCandidateDirectory @@ -135,6 +136,21 @@ describe('retained release candidate', () => { expect(rejected.bundlePath).toBe(prepared.bundlePath) }) + it('permits verified published candidates only for an explicit publish retry', async () => { + await materializeReleaseCandidate({ bundlePath: await createBundle() }) + await recordCandidateE2E({ tag: 'windows-v0.1.21', platform: 'windows-x64', status: 'passed' }) + await markCandidatePublished({ tag: 'windows-v0.1.21', platform: 'windows-x64' }) + const request = { + tag: 'windows-v0.1.21', + platform: 'windows-x64', + commit: '0123456789012345678901234567890123456789' + } + await expect(assertCandidatePublishable(request)).rejects.toThrow('is published') + await expect( + assertCandidatePublishable({ ...request, allowPublished: true }) + ).resolves.toBeDefined() + }) + it('allows an explicitly justified E2E skip to become publishable', async () => { const bundlePath = await createBundle() await materializeReleaseCandidate({ bundlePath }) diff --git a/tests/case/unit/release/live-tag.test.ts b/tests/case/unit/release/live-tag.test.ts index c8e370ab..bcc9acd5 100644 --- a/tests/case/unit/release/live-tag.test.ts +++ b/tests/case/unit/release/live-tag.test.ts @@ -1,5 +1,8 @@ import { describe, expect, it } from 'vitest' -import { findLiveChannelsForTag } from '../../../../scripts/release/shared/live-tag.mjs' +import { + findLiveChannelsForTag, + isReleaseChannelCurrent +} from '../../../../scripts/release/shared/live-tag.mjs' describe('release live tag detection', () => { it('reports channels whose latest manifest points at the tag', async () => { @@ -27,4 +30,72 @@ describe('release live tag detection', () => { }) await expect(findLiveChannelsForTag('mac-v1.0.0', fetchImpl)).resolves.toEqual([]) }) + + it.each(['windows', 'mac'] as const)( + 'verifies the %s manifest and relative update URL against the candidate', + async (family) => { + const base = 'https://download.guanzhangai.cn' + const tag = `${family}-v1.0.0` + const folder = family === 'windows' ? 'windows' : 'macos' + const installerName = + family === 'windows' ? '馆长-1.0.0-Windows-x64.exe' : '馆长-1.0.0-macos-arm64.dmg' + const installer = { + role: 'installer', + name: installerName, + size: 123, + sha512: 'installer-hash' + } + const update = + family === 'windows' + ? installer + : { role: 'update', name: 'Ncurator-mac-arm64.zip', size: 456, sha512: 'zip-hash' } + const fragment = { + tag, + platform: family === 'windows' ? 'windows-x64' : 'macos-arm64', + files: [installer, ...(family === 'mac' ? [update] : [])] + } + const manifest = { + version: '1.0.0', + tag, + downloads: { + [family === 'windows' ? 'x64' : 'arm64']: { + ...installer, + url: `${base}/artifacts/${folder}/${tag}/${installerName}` + } + } + } + let metadata = `version: 1.0.0\nfiles:\n - url: ../../artifacts/${folder}/${tag}/${update.name}\n size: ${update.size}\n sha512: ${update.sha512}\n` + const fetchImpl = async (url: string) => + new Response(url.includes('.json') ? JSON.stringify(manifest) : metadata) + await expect(isReleaseChannelCurrent(fragment, base, fetchImpl)).resolves.toBe(true) + metadata = metadata.replace(`/${tag}/`, '/old-tag/') + await expect(isReleaseChannelCurrent(fragment, base, fetchImpl)).resolves.toBe(false) + metadata = metadata.replace('version: 1.0.0', 'version: 1.0.1') + await expect(isReleaseChannelCurrent(fragment, base, fetchImpl)).rejects.toThrow( + 'refuse to downgrade' + ) + } + ) + + it('distinguishes an absent channel from a failed lookup', async () => { + const fragment = { + tag: 'windows-v1.0.0', + platform: 'windows-x64', + files: [{ role: 'installer', name: 'installer.exe', size: 123, sha512: 'hash' }] + } + await expect( + isReleaseChannelCurrent( + fragment, + 'https://example.com', + async () => new Response(null, { status: 404 }) + ) + ).resolves.toBe(false) + await expect( + isReleaseChannelCurrent( + fragment, + 'https://example.com', + async () => new Response(null, { status: 503 }) + ) + ).rejects.toThrow('HTTP 503') + }) })