refactor(engine): 将 engine 从 submodule 内嵌为仓内一等组件
`vendor/MetasequoiaImeEngine` submodule 改为 `engine/` 目录,导入的是 product-lock.json 原本钉住的提交 c810d201,逐文件核对过 blob,因此这是一次 搬迁而不是引擎升级。引擎自己的嵌套 submodule(googlepinyinime-rev、utfcpp、 miniaudio)按各自 gitlink 一并展开,来源与提交见 engine/UPSTREAM.md。 Windows 专门化裁掉了不属于本产品的部分:whisper.cpp 与本地推理 provider、 macOS 示例宿主、整个 dictionary/ 建库流水线、上游 CI 与 release-please 配置。 词库仍按 product-lock.json 在构建时取发布产物。 引擎不再进 product-lock.json:没有 gitlink 就没有可比对的东西,锁里写一个上游 commit 等于宣称这棵树仍等于上游,而第一个 Windows 专门化改动落地时这句话就假了。 verify-contracts / verify-checkout 随之移除,verify-published 保留。 CI 与工具链:引擎源码并入根 scripts/format.sh(删掉引擎自己的那份,避免两套排除 列表分叉);引擎 voice.yml 里的 silero_vad.onnx 摘要校验移到本仓 lock job; engine-bump-triage 工作流与其测试删除;gitsubmodule 的 dependabot 频率由 daily 改为 monthly。 README、README.en、AGENTS.md、CONTRIBUTING、docs/product-release.md 及各子目录 文档同步说明新的位置与边界。
This commit is contained in:
1 parent
987f404343
commit
a7779d2b05
406 files changed
+212487
-541
No files matched your search
@@ -7,9 +7,12 @@ contact_links:
|
||||
- name: 安全漏洞 / Security vulnerability
|
||||
url: https://github.com/metasequoiaime/MSIME-Windows/security/advisories/new
|
||||
about: 疑似漏洞请私下上报,不要提交公开 issue。Report privately; do not open a public issue.
|
||||
- name: 组词引擎与词库问题 / Engine and dictionary
|
||||
# The engine moved in-tree, so candidate-order bugs are now this repository's own bugs and belong
|
||||
# in the normal bug form. The dictionary data is still built and released elsewhere, so a missing
|
||||
# word is still someone else's to fix -- that is the only part this row still routes away.
|
||||
- name: 词库缺词 / Missing dictionary words
|
||||
url: https://github.com/metasequoiaime/MSIME-Engine/issues
|
||||
about: 选字顺序、词库缺词等属于共享引擎与词库,请到对应仓库反馈。Candidate order and missing words belong to the shared engine.
|
||||
about: 词库数据在仓外构建发布,缺词、错词请到对应仓库反馈;选字顺序等引擎问题请直接在本仓提 bug。Dictionary data is built elsewhere; engine behaviour belongs here.
|
||||
- name: Telegram 群组 / Telegram Group
|
||||
url: https://t.me/msimegroup
|
||||
about: 使用问题、讨论与日常交流。Questions and discussions.
|
||||
|
||||
@@ -12,13 +12,14 @@ updates:
|
||||
prefix: "chore(deps)"
|
||||
open-pull-requests-limit: 5
|
||||
|
||||
# Daily rather than monthly, which is the cadence Server used before the consolidation. The
|
||||
# engine moves far faster than the frontends and engine-bump-triage.yml exists to absorb that
|
||||
# rate: it auto-merges the provably inert bumps and holds the rest for review.
|
||||
# Daily was the engine's cadence: it moved far faster than the frontends, and engine-bump-triage.yml
|
||||
# absorbed that rate by auto-merging the provably inert bumps. The engine is vendored in-tree now,
|
||||
# so what is left here is opencc and cpp-pinyin -- third-party libraries that release a few times a
|
||||
# year. A daily poll on those is a daily no-op.
|
||||
- package-ecosystem: "gitsubmodule"
|
||||
directory: "/"
|
||||
schedule:
|
||||
interval: "daily"
|
||||
interval: "monthly"
|
||||
commit-message:
|
||||
prefix: "chore(deps)"
|
||||
open-pull-requests-limit: 5
|
||||
|
||||
+13
-15
@@ -24,30 +24,28 @@ jobs:
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 60
|
||||
steps:
|
||||
# The dictionary validator is compared against the engine's copy of it, so this job needs the
|
||||
# submodule's contents and not just its gitlink.
|
||||
# The engine is vendored in-tree, so a plain checkout already carries the contracts this job
|
||||
# compares against. opencc and cpp-pinyin are still submodules but nothing here builds.
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
submodules: true
|
||||
- name: Check the shared dictionary validator
|
||||
run: python3 scripts/check-dictionary-contract.py
|
||||
- name: Check product-lock contract
|
||||
run: python3 scripts/check-product-lock-contract.py
|
||||
# The voice activity detector is committed rather than downloaded, and it is executed inside
|
||||
# the IME process. The engine's own voice.yml used to re-check its digest; that workflow left
|
||||
# with the submodule, so the check moves here rather than disappearing.
|
||||
- name: Check the committed voice model against its recorded digest
|
||||
run: |
|
||||
echo '1a153a22f4509e292a94e67d6f9b85e8deb25b4988682b7e174c65279d8788e3 engine/voice/assets/models/silero_vad.onnx' | sha256sum --check --strict
|
||||
- name: Test rejection of changed or mismatched inputs
|
||||
run: python3 -m unittest discover -s tests -p 'test_product_lock.py'
|
||||
- name: Test local environment setup in a moved checkout
|
||||
run: python3 -m unittest discover -s tests -p 'test_prepare_env.py'
|
||||
- name: Test the automatic release CI gate
|
||||
run: python3 -m unittest discover -s tests -p 'test_release_pr_ci.py'
|
||||
- name: Test the engine-bump auto-merge guards
|
||||
run: |
|
||||
python3 -m pip install --quiet pyyaml
|
||||
python3 -m unittest discover -s tests -p 'test_engine_bump_triage.py'
|
||||
- name: Read the reviewed lock
|
||||
id: lock
|
||||
run: |
|
||||
python3 scripts/product_lock.py verify-contracts .
|
||||
python3 scripts/product_lock.py outputs --github-output "$GITHUB_OUTPUT"
|
||||
run: python3 scripts/product_lock.py outputs --github-output "$GITHUB_OUTPUT"
|
||||
|
||||
windows:
|
||||
name: Windows ${{ matrix.name }}
|
||||
@@ -231,12 +229,12 @@ jobs:
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
persist-credentials: false
|
||||
submodules: true
|
||||
|
||||
# The pages ship a copy of the engine's web contract. Before the consolidation the product
|
||||
# lock kept that copy honest by pinning UiHtml and the engine together; inside one repository
|
||||
# the engine submodule is the pin, so the copy is checked against it directly.
|
||||
- name: Check the bundled contract against the engine submodule
|
||||
# lock kept that copy honest by pinning UiHtml and the engine together; now that the engine is
|
||||
# a directory of this repository, the copy is checked against engine/contracts/webview
|
||||
# directly and a plain checkout is enough to do it.
|
||||
- name: Check the bundled contract against the engine
|
||||
run: python3 ui-html/scripts/sync-contracts.py --check
|
||||
|
||||
- name: Set up pnpm
|
||||
|
||||
@@ -1,169 +0,0 @@
|
||||
name: Engine bump triage
|
||||
|
||||
# Dependabot opens the engine submodule bump; this decides whether it can merge itself.
|
||||
#
|
||||
# The rule is an allowlist, not a blocklist: the bump merges on green CI only when every file the
|
||||
# engine changed across the range is provably inert. Anything else waits for a human. A blocklist
|
||||
# would have let the utfcpp bump through, because it appears in the range as a bare submodule
|
||||
# pointer rather than as a source file.
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
paths:
|
||||
- 'vendor/MetasequoiaImeEngine'
|
||||
|
||||
# Read-only by default, matching every other workflow here. The write scopes this job needs to merge
|
||||
# the bump are granted on the job instead, so a second job added later starts from the same floor
|
||||
# rather than inheriting write.
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
triage:
|
||||
if: github.actor == 'dependabot[bot]'
|
||||
name: Classify the engine range
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 10
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
env:
|
||||
SUBMODULE: vendor/MetasequoiaImeEngine
|
||||
ENGINE: metasequoiaime/MSIME-Engine
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
PR_URL: ${{ github.event.pull_request.html_url }}
|
||||
steps:
|
||||
- name: Check out source
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
- name: Resolve the engine range
|
||||
id: range
|
||||
env:
|
||||
BASE_REF: ${{ github.base_ref }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
base=$(git rev-parse "origin/$BASE_REF:$SUBMODULE")
|
||||
head=$(git rev-parse "HEAD:$SUBMODULE")
|
||||
echo "base=$base" >> "$GITHUB_OUTPUT"
|
||||
echo "head=$head" >> "$GITHUB_OUTPUT"
|
||||
echo "引擎 $base -> $head"
|
||||
|
||||
- name: Classify the changed files
|
||||
id: classify
|
||||
env:
|
||||
BASE: ${{ steps.range.outputs.base }}
|
||||
HEAD: ${{ steps.range.outputs.head }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
base="$BASE"
|
||||
head="$HEAD"
|
||||
if [ "$base" = "$head" ]; then
|
||||
echo "verdict=inert" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Fail closed on anything but a plain fast-forward. The compare API answers with an empty
|
||||
# file list whenever head is already an ancestor of base, which the emptiness test below
|
||||
# would otherwise read as "nothing suspicious changed" and auto-merge — silently accepting
|
||||
# a downgrade that reverts engine source the frontend depends on.
|
||||
status=$(gh api "repos/$ENGINE/compare/$base...$head" --jq '.status')
|
||||
echo "compare status: $status"
|
||||
if [ "$status" != "ahead" ]; then
|
||||
echo "verdict=review" >> "$GITHUB_OUTPUT"
|
||||
delimiter="EOF_$(openssl rand -hex 16)"
|
||||
{
|
||||
echo "reason<<$delimiter"
|
||||
echo "引擎范围不是单纯的快进(compare status: $status),需要人工确认。"
|
||||
echo "$delimiter"
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
files=$(gh api --paginate "repos/$ENGINE/compare/$base...$head" --jq '.files[].filename')
|
||||
echo "改动文件:"; echo "$files" | sed 's/^/ /'
|
||||
if [ -z "$files" ]; then
|
||||
echo "verdict=review" >> "$GITHUB_OUTPUT"
|
||||
delimiter="EOF_$(openssl rand -hex 16)"
|
||||
{
|
||||
echo "reason<<$delimiter"
|
||||
echo "compare 没有返回任何文件,无法判定这次 bump 是否安全。"
|
||||
echo "$delimiter"
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Provably inert: documentation, CI wiring, dependency manifests, licence text. Everything
|
||||
# else - sources, build files, and bare paths such as a nested submodule pointer - needs a
|
||||
# human. Note the trailing anchors: 'utfcpp' matches none of these patterns and so lands
|
||||
# in review, which is the behaviour we want.
|
||||
suspicious=$(echo "$files" | grep -vE '(^|/)[^/]+\.md$|^\.github/|^vcpkg\.json$|^vcpkg-configuration\.json$|^LICENSE$|^\.gitignore$' || true)
|
||||
|
||||
if [ -z "$suspicious" ]; then
|
||||
echo "verdict=inert" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "verdict=review" >> "$GITHUB_OUTPUT"
|
||||
# A random delimiter: the file names come from another repository, so a path that
|
||||
# happened to be the literal delimiter could otherwise close the block early.
|
||||
delimiter="EOF_$(openssl rand -hex 16)"
|
||||
{
|
||||
echo "reason<<$delimiter"
|
||||
echo "$suspicious"
|
||||
echo "$delimiter"
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
# The product lock records the engine commit for provenance and for the release gate that
|
||||
# refuses a commit nobody merged. A bump that moves the submodule without the lock leaves the
|
||||
# two disagreeing, and verify-contracts fails the whole product-inputs job rather than this
|
||||
# pull request, so say plainly which file has to move.
|
||||
- name: Check the lock followed the submodule
|
||||
id: lock
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if python3 scripts/product_lock.py verify-contracts .; then
|
||||
echo "locked=yes" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "locked=no" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
- name: Enable auto-merge
|
||||
# The lock guard belongs here. It used to be appended to the HEAD environment value instead,
|
||||
# where it was not a condition at all: auto-merge ran whether or not the contract lock
|
||||
# verified, and the comment below printed the commit followed by the literal text
|
||||
# "&& steps.lock.outputs.locked == 'yes'".
|
||||
if: steps.classify.outputs.verdict == 'inert' && steps.lock.outputs.locked == 'yes'
|
||||
env:
|
||||
BASE: ${{ steps.range.outputs.base }}
|
||||
HEAD: ${{ steps.range.outputs.head }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
gh pr merge --auto --squash "$PR_URL"
|
||||
body="$RUNNER_TEMP/engine-bump-comment.md"
|
||||
printf '引擎 `%s` → `%s` 的改动全部落在文档、CI 配置和依赖清单里,没有源码、构建文件或嵌套 submodule 指针。已开启自动合并,CI 通过后自行合入。\n' "$BASE" "$HEAD" > "$body"
|
||||
gh pr comment "$PR_URL" --body-file "$body"
|
||||
|
||||
# Every value below reaches the shell through the environment, never through ${{ }}
|
||||
# interpolation. REASON is a list of file names from another repository: spliced into the
|
||||
# command text, a path containing a single quote would have closed the quoting and run as a
|
||||
# command, on a runner holding a contents:write token for this repository.
|
||||
- name: Hold for review
|
||||
if: steps.classify.outputs.verdict == 'review'
|
||||
env:
|
||||
BASE: ${{ steps.range.outputs.base }}
|
||||
HEAD: ${{ steps.range.outputs.head }}
|
||||
REASON: ${{ steps.classify.outputs.reason }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
gh pr edit "$PR_URL" --add-label "needs-review" || true
|
||||
body="$RUNNER_TEMP/engine-bump-comment.md"
|
||||
printf '引擎 `%s` → `%s` 的改动里有需要人看一眼的部分,未开启自动合并:\n\n```\n%s\n```\n\n源码、`CMakeLists.txt` 或嵌套 submodule 指针的变化都可能改变行为,而前端 CI 未必覆盖得到——`helpcode_enabled()` 与 `handle_character()` 的语义变更就是这样漏过去的。\n' "$BASE" "$HEAD" "$REASON" > "$body"
|
||||
gh pr comment "$PR_URL" --body-file "$body"
|
||||
|
||||
- name: Ask for the lock to follow
|
||||
if: steps.lock.outputs.locked == 'no'
|
||||
run: |
|
||||
set -euo pipefail
|
||||
gh pr comment "$PR_URL" --body "引擎 submodule 动了但 \`product-lock.json\` 里的 \`engine.commit\` 没跟上,产物清单会记录错误的引擎来源,发布门禁也拦不住未合入的引擎提交。跑 \`python3 scripts/product_lock.py refresh --dictionary-tag <当前 tag>\` 或直接改这一个字段,再推上来。"
|
||||
@@ -146,7 +146,6 @@ jobs:
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
run: |
|
||||
python3 scripts/product_lock.py verify-contracts .
|
||||
python3 scripts/product_lock.py verify-published
|
||||
python3 scripts/product_lock.py outputs --github-output "$GITHUB_OUTPUT"
|
||||
|
||||
@@ -347,7 +346,6 @@ jobs:
|
||||
with:
|
||||
ref: ${{ needs.prepare.outputs.target_sha }}
|
||||
persist-credentials: false
|
||||
submodules: true
|
||||
|
||||
- name: Verify the shipped pages carry the engine's contract
|
||||
run: python3 ui-html/scripts/sync-contracts.py --check
|
||||
@@ -407,13 +405,12 @@ jobs:
|
||||
# outside, so it lands beside the components in the one checkout.
|
||||
STAGING_ROOT: .
|
||||
steps:
|
||||
# The helpcodes are collected out of the engine submodule now, so this job needs its contents
|
||||
# and not just the gitlink.
|
||||
# The helpcodes are collected out of engine/helpcode, which is a directory of this repository,
|
||||
# so a plain checkout of the release commit already carries them.
|
||||
- name: Check out the release commit
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
ref: ${{ needs.prepare.outputs.target_sha }}
|
||||
submodules: true
|
||||
persist-credentials: false
|
||||
|
||||
# Some runner instances leave _work owned by SYSTEM while the job itself
|
||||
@@ -489,7 +486,7 @@ jobs:
|
||||
pwsh -File ./Prepare-PackageFiles.ps1 -TargetVersion $env:TARGET_VERSION
|
||||
-RepoRoot (Resolve-Path '..').Path
|
||||
-TsfDirectory windows -ServerDirectory server -UiHtmlDirectory ui-html -NoticesDirectory .
|
||||
-HelpCodeDirectory vendor/MetasequoiaImeEngine/helpcode
|
||||
-HelpCodeDirectory engine/helpcode
|
||||
|
||||
- name: Embed the exact product dependency manifest
|
||||
env:
|
||||
|
||||
Reference in new issue
Block a user