refactor(engine): 将 engine 从 submodule 内嵌为仓内一等组件

`vendor/MetasequoiaImeEngine` submodule 改为 `engine/` 目录,导入的是
product-lock.json 原本钉住的提交 c810d201,逐文件核对过 blob,因此这是一次
搬迁而不是引擎升级。引擎自己的嵌套 submodule(googlepinyinime-rev、utfcpp、
miniaudio)按各自 gitlink 一并展开,来源与提交见 engine/UPSTREAM.md。

Windows 专门化裁掉了不属于本产品的部分:whisper.cpp 与本地推理 provider、
macOS 示例宿主、整个 dictionary/ 建库流水线、上游 CI 与 release-please 配置。
词库仍按 product-lock.json 在构建时取发布产物。

引擎不再进 product-lock.json:没有 gitlink 就没有可比对的东西,锁里写一个上游
commit 等于宣称这棵树仍等于上游,而第一个 Windows 专门化改动落地时这句话就假了。
verify-contracts / verify-checkout 随之移除,verify-published 保留。

CI 与工具链:引擎源码并入根 scripts/format.sh(删掉引擎自己的那份,避免两套排除
列表分叉);引擎 voice.yml 里的 silero_vad.onnx 摘要校验移到本仓 lock job;
engine-bump-triage 工作流与其测试删除;gitsubmodule 的 dependabot 频率由 daily
改为 monthly。

README、README.en、AGENTS.md、CONTRIBUTING、docs/product-release.md 及各子目录
文档同步说明新的位置与边界。
This commit is contained in:
fanlusky committed 2026-09-12 03:40:48 +08:00
1 parent 987f404343
commit a7779d2b05
406 files changed
+212487 -541

No files matched your search

+5 -2
View File
@@ -7,9 +7,12 @@ contact_links:
- name: 安全漏洞 / Security vulnerability
url: https://github.com/metasequoiaime/MSIME-Windows/security/advisories/new
about: 疑似漏洞请私下上报,不要提交公开 issue。Report privately; do not open a public issue.
- name: 组词引擎与词库问题 / Engine and dictionary
# The engine moved in-tree, so candidate-order bugs are now this repository's own bugs and belong
# in the normal bug form. The dictionary data is still built and released elsewhere, so a missing
# word is still someone else's to fix -- that is the only part this row still routes away.
- name: 词库缺词 / Missing dictionary words
url: https://github.com/metasequoiaime/MSIME-Engine/issues
about: 选字顺序、词库缺词等属于共享引擎与词库,请到对应仓库反馈。Candidate order and missing words belong to the shared engine.
about: 词库数据在仓外构建发布,缺词、错词请到对应仓库反馈;选字顺序等引擎问题请直接在本仓提 bug。Dictionary data is built elsewhere; engine behaviour belongs here.
- name: Telegram 群组 / Telegram Group
url: https://t.me/msimegroup
about: 使用问题、讨论与日常交流。Questions and discussions.
+5 -4
View File
@@ -12,13 +12,14 @@ updates:
prefix: "chore(deps)"
open-pull-requests-limit: 5
# Daily rather than monthly, which is the cadence Server used before the consolidation. The
# engine moves far faster than the frontends and engine-bump-triage.yml exists to absorb that
# rate: it auto-merges the provably inert bumps and holds the rest for review.
# Daily was the engine's cadence: it moved far faster than the frontends, and engine-bump-triage.yml
# absorbed that rate by auto-merging the provably inert bumps. The engine is vendored in-tree now,
# so what is left here is opencc and cpp-pinyin -- third-party libraries that release a few times a
# year. A daily poll on those is a daily no-op.
- package-ecosystem: "gitsubmodule"
directory: "/"
schedule:
interval: "daily"
interval: "monthly"
commit-message:
prefix: "chore(deps)"
open-pull-requests-limit: 5
+13 -15
View File
@@ -24,30 +24,28 @@ jobs:
runs-on: ubuntu-24.04
timeout-minutes: 60
steps:
# The dictionary validator is compared against the engine's copy of it, so this job needs the
# submodule's contents and not just its gitlink.
# The engine is vendored in-tree, so a plain checkout already carries the contracts this job
# compares against. opencc and cpp-pinyin are still submodules but nothing here builds.
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
submodules: true
- name: Check the shared dictionary validator
run: python3 scripts/check-dictionary-contract.py
- name: Check product-lock contract
run: python3 scripts/check-product-lock-contract.py
# The voice activity detector is committed rather than downloaded, and it is executed inside
# the IME process. The engine's own voice.yml used to re-check its digest; that workflow left
# with the submodule, so the check moves here rather than disappearing.
- name: Check the committed voice model against its recorded digest
run: |
echo '1a153a22f4509e292a94e67d6f9b85e8deb25b4988682b7e174c65279d8788e3 engine/voice/assets/models/silero_vad.onnx' | sha256sum --check --strict
- name: Test rejection of changed or mismatched inputs
run: python3 -m unittest discover -s tests -p 'test_product_lock.py'
- name: Test local environment setup in a moved checkout
run: python3 -m unittest discover -s tests -p 'test_prepare_env.py'
- name: Test the automatic release CI gate
run: python3 -m unittest discover -s tests -p 'test_release_pr_ci.py'
- name: Test the engine-bump auto-merge guards
run: |
python3 -m pip install --quiet pyyaml
python3 -m unittest discover -s tests -p 'test_engine_bump_triage.py'
- name: Read the reviewed lock
id: lock
run: |
python3 scripts/product_lock.py verify-contracts .
python3 scripts/product_lock.py outputs --github-output "$GITHUB_OUTPUT"
run: python3 scripts/product_lock.py outputs --github-output "$GITHUB_OUTPUT"
windows:
name: Windows ${{ matrix.name }}
@@ -231,12 +229,12 @@ jobs:
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
submodules: true
# The pages ship a copy of the engine's web contract. Before the consolidation the product
# lock kept that copy honest by pinning UiHtml and the engine together; inside one repository
# the engine submodule is the pin, so the copy is checked against it directly.
- name: Check the bundled contract against the engine submodule
# lock kept that copy honest by pinning UiHtml and the engine together; now that the engine is
# a directory of this repository, the copy is checked against engine/contracts/webview
# directly and a plain checkout is enough to do it.
- name: Check the bundled contract against the engine
run: python3 ui-html/scripts/sync-contracts.py --check
- name: Set up pnpm
-169
View File
@@ -1,169 +0,0 @@
name: Engine bump triage
# Dependabot opens the engine submodule bump; this decides whether it can merge itself.
#
# The rule is an allowlist, not a blocklist: the bump merges on green CI only when every file the
# engine changed across the range is provably inert. Anything else waits for a human. A blocklist
# would have let the utfcpp bump through, because it appears in the range as a bare submodule
# pointer rather than as a source file.
on:
pull_request:
paths:
- 'vendor/MetasequoiaImeEngine'
# Read-only by default, matching every other workflow here. The write scopes this job needs to merge
# the bump are granted on the job instead, so a second job added later starts from the same floor
# rather than inheriting write.
permissions:
contents: read
jobs:
triage:
if: github.actor == 'dependabot[bot]'
name: Classify the engine range
runs-on: ubuntu-24.04
timeout-minutes: 10
permissions:
contents: write
pull-requests: write
env:
SUBMODULE: vendor/MetasequoiaImeEngine
ENGINE: metasequoiaime/MSIME-Engine
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_URL: ${{ github.event.pull_request.html_url }}
steps:
- name: Check out source
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0
persist-credentials: false
- name: Resolve the engine range
id: range
env:
BASE_REF: ${{ github.base_ref }}
run: |
set -euo pipefail
base=$(git rev-parse "origin/$BASE_REF:$SUBMODULE")
head=$(git rev-parse "HEAD:$SUBMODULE")
echo "base=$base" >> "$GITHUB_OUTPUT"
echo "head=$head" >> "$GITHUB_OUTPUT"
echo "引擎 $base -> $head"
- name: Classify the changed files
id: classify
env:
BASE: ${{ steps.range.outputs.base }}
HEAD: ${{ steps.range.outputs.head }}
run: |
set -euo pipefail
base="$BASE"
head="$HEAD"
if [ "$base" = "$head" ]; then
echo "verdict=inert" >> "$GITHUB_OUTPUT"
exit 0
fi
# Fail closed on anything but a plain fast-forward. The compare API answers with an empty
# file list whenever head is already an ancestor of base, which the emptiness test below
# would otherwise read as "nothing suspicious changed" and auto-merge — silently accepting
# a downgrade that reverts engine source the frontend depends on.
status=$(gh api "repos/$ENGINE/compare/$base...$head" --jq '.status')
echo "compare status: $status"
if [ "$status" != "ahead" ]; then
echo "verdict=review" >> "$GITHUB_OUTPUT"
delimiter="EOF_$(openssl rand -hex 16)"
{
echo "reason<<$delimiter"
echo "引擎范围不是单纯的快进(compare status: $status),需要人工确认。"
echo "$delimiter"
} >> "$GITHUB_OUTPUT"
exit 0
fi
files=$(gh api --paginate "repos/$ENGINE/compare/$base...$head" --jq '.files[].filename')
echo "改动文件:"; echo "$files" | sed 's/^/ /'
if [ -z "$files" ]; then
echo "verdict=review" >> "$GITHUB_OUTPUT"
delimiter="EOF_$(openssl rand -hex 16)"
{
echo "reason<<$delimiter"
echo "compare 没有返回任何文件,无法判定这次 bump 是否安全。"
echo "$delimiter"
} >> "$GITHUB_OUTPUT"
exit 0
fi
# Provably inert: documentation, CI wiring, dependency manifests, licence text. Everything
# else - sources, build files, and bare paths such as a nested submodule pointer - needs a
# human. Note the trailing anchors: 'utfcpp' matches none of these patterns and so lands
# in review, which is the behaviour we want.
suspicious=$(echo "$files" | grep -vE '(^|/)[^/]+\.md$|^\.github/|^vcpkg\.json$|^vcpkg-configuration\.json$|^LICENSE$|^\.gitignore$' || true)
if [ -z "$suspicious" ]; then
echo "verdict=inert" >> "$GITHUB_OUTPUT"
else
echo "verdict=review" >> "$GITHUB_OUTPUT"
# A random delimiter: the file names come from another repository, so a path that
# happened to be the literal delimiter could otherwise close the block early.
delimiter="EOF_$(openssl rand -hex 16)"
{
echo "reason<<$delimiter"
echo "$suspicious"
echo "$delimiter"
} >> "$GITHUB_OUTPUT"
fi
# The product lock records the engine commit for provenance and for the release gate that
# refuses a commit nobody merged. A bump that moves the submodule without the lock leaves the
# two disagreeing, and verify-contracts fails the whole product-inputs job rather than this
# pull request, so say plainly which file has to move.
- name: Check the lock followed the submodule
id: lock
run: |
set -euo pipefail
if python3 scripts/product_lock.py verify-contracts .; then
echo "locked=yes" >> "$GITHUB_OUTPUT"
else
echo "locked=no" >> "$GITHUB_OUTPUT"
fi
- name: Enable auto-merge
# The lock guard belongs here. It used to be appended to the HEAD environment value instead,
# where it was not a condition at all: auto-merge ran whether or not the contract lock
# verified, and the comment below printed the commit followed by the literal text
# "&& steps.lock.outputs.locked == 'yes'".
if: steps.classify.outputs.verdict == 'inert' && steps.lock.outputs.locked == 'yes'
env:
BASE: ${{ steps.range.outputs.base }}
HEAD: ${{ steps.range.outputs.head }}
run: |
set -euo pipefail
gh pr merge --auto --squash "$PR_URL"
body="$RUNNER_TEMP/engine-bump-comment.md"
printf '引擎 `%s` → `%s` 的改动全部落在文档、CI 配置和依赖清单里,没有源码、构建文件或嵌套 submodule 指针。已开启自动合并,CI 通过后自行合入。\n' "$BASE" "$HEAD" > "$body"
gh pr comment "$PR_URL" --body-file "$body"
# Every value below reaches the shell through the environment, never through ${{ }}
# interpolation. REASON is a list of file names from another repository: spliced into the
# command text, a path containing a single quote would have closed the quoting and run as a
# command, on a runner holding a contents:write token for this repository.
- name: Hold for review
if: steps.classify.outputs.verdict == 'review'
env:
BASE: ${{ steps.range.outputs.base }}
HEAD: ${{ steps.range.outputs.head }}
REASON: ${{ steps.classify.outputs.reason }}
run: |
set -euo pipefail
gh pr edit "$PR_URL" --add-label "needs-review" || true
body="$RUNNER_TEMP/engine-bump-comment.md"
printf '引擎 `%s` → `%s` 的改动里有需要人看一眼的部分,未开启自动合并:\n\n```\n%s\n```\n\n源码、`CMakeLists.txt` 或嵌套 submodule 指针的变化都可能改变行为,而前端 CI 未必覆盖得到——`helpcode_enabled()` 与 `handle_character()` 的语义变更就是这样漏过去的。\n' "$BASE" "$HEAD" "$REASON" > "$body"
gh pr comment "$PR_URL" --body-file "$body"
- name: Ask for the lock to follow
if: steps.lock.outputs.locked == 'no'
run: |
set -euo pipefail
gh pr comment "$PR_URL" --body "引擎 submodule 动了但 \`product-lock.json\` 里的 \`engine.commit\` 没跟上,产物清单会记录错误的引擎来源,发布门禁也拦不住未合入的引擎提交。跑 \`python3 scripts/product_lock.py refresh --dictionary-tag <当前 tag>\` 或直接改这一个字段,再推上来。"
+3 -6
View File
@@ -146,7 +146,6 @@ jobs:
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
python3 scripts/product_lock.py verify-contracts .
python3 scripts/product_lock.py verify-published
python3 scripts/product_lock.py outputs --github-output "$GITHUB_OUTPUT"
@@ -347,7 +346,6 @@ jobs:
with:
ref: ${{ needs.prepare.outputs.target_sha }}
persist-credentials: false
submodules: true
- name: Verify the shipped pages carry the engine's contract
run: python3 ui-html/scripts/sync-contracts.py --check
@@ -407,13 +405,12 @@ jobs:
# outside, so it lands beside the components in the one checkout.
STAGING_ROOT: .
steps:
# The helpcodes are collected out of the engine submodule now, so this job needs its contents
# and not just the gitlink.
# The helpcodes are collected out of engine/helpcode, which is a directory of this repository,
# so a plain checkout of the release commit already carries them.
- name: Check out the release commit
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ needs.prepare.outputs.target_sha }}
submodules: true
persist-credentials: false
# Some runner instances leave _work owned by SYSTEM while the job itself
@@ -489,7 +486,7 @@ jobs:
pwsh -File ./Prepare-PackageFiles.ps1 -TargetVersion $env:TARGET_VERSION
-RepoRoot (Resolve-Path '..').Path
-TsfDirectory windows -ServerDirectory server -UiHtmlDirectory ui-html -NoticesDirectory .
-HelpCodeDirectory vendor/MetasequoiaImeEngine/helpcode
-HelpCodeDirectory engine/helpcode
- name: Embed the exact product dependency manifest
env: