refactor(engine): 将 engine 从 submodule 内嵌为仓内一等组件
`vendor/MetasequoiaImeEngine` submodule 改为 `engine/` 目录,导入的是 product-lock.json 原本钉住的提交 c810d201,逐文件核对过 blob,因此这是一次 搬迁而不是引擎升级。引擎自己的嵌套 submodule(googlepinyinime-rev、utfcpp、 miniaudio)按各自 gitlink 一并展开,来源与提交见 engine/UPSTREAM.md。 Windows 专门化裁掉了不属于本产品的部分:whisper.cpp 与本地推理 provider、 macOS 示例宿主、整个 dictionary/ 建库流水线、上游 CI 与 release-please 配置。 词库仍按 product-lock.json 在构建时取发布产物。 引擎不再进 product-lock.json:没有 gitlink 就没有可比对的东西,锁里写一个上游 commit 等于宣称这棵树仍等于上游,而第一个 Windows 专门化改动落地时这句话就假了。 verify-contracts / verify-checkout 随之移除,verify-published 保留。 CI 与工具链:引擎源码并入根 scripts/format.sh(删掉引擎自己的那份,避免两套排除 列表分叉);引擎 voice.yml 里的 silero_vad.onnx 摘要校验移到本仓 lock job; engine-bump-triage 工作流与其测试删除;gitsubmodule 的 dependabot 频率由 daily 改为 monthly。 README、README.en、AGENTS.md、CONTRIBUTING、docs/product-release.md 及各子目录 文档同步说明新的位置与边界。
This commit is contained in:
1 parent
987f404343
commit
a7779d2b05
406 files changed
+212487
-541
No files matched your search
@@ -0,0 +1,25 @@
|
||||
# 来源与授权说明
|
||||
|
||||
本仓库**不对外提供统一的开源许可**。`helpcodes/` 下的辅助码表复现的是各家已发表的输入方案,权利归各方案作者,本项目只是整理成统一格式供引擎读取。给整个仓库挂一份 LICENSE 等于替方案作者重新授权,因此这里改为逐项说明来源。
|
||||
|
||||
## 辅助码表
|
||||
|
||||
| 文件 | 方案 | 来源与权利归属 |
|
||||
| --- | --- | --- |
|
||||
| `helpcode.txt` | `lantian` | 蓝天小雨点辅助码,权利归方案作者 |
|
||||
| `zrm_helpcode_big_unique.txt` | `ziranma` | 自然码辅助码,整理自 [copperay/ZRM_Aux-code](https://github.com/copperay/ZRM_Aux-code)(上游**未声明许可**) |
|
||||
| `shouyou2_0_helpcode.txt` | `shouyou2_0` | 首右 2.0 辅助码,权利归方案作者 |
|
||||
| `shouyouplus_helpcode.txt` | `shouyouplus` | 首右 plus 辅助码,权利归方案作者 |
|
||||
| `xiaohe_helpcode.txt` | `xiaohe` | 小鹤形码,权利归小鹤方案作者 |
|
||||
|
||||
## 下游影响
|
||||
|
||||
这些文件被 [MSIME-Linux](https://github.com/metasequoiaime/MSIME-Linux) 的 DEB/RPM 包安装到 `share/metasequoiaime/helpcodes/` 下,也被 Windows 与 Apple 前端使用。前端本身以 GPL-3.0 分发,但该许可**不覆盖**这些辅助码表的内容。
|
||||
|
||||
## 待解决
|
||||
|
||||
上表中没有任何一项拿到了明确的再分发授权。需要逐个与方案作者确认,或改为在运行时由用户自行导入而不随包分发。在澄清之前,请不要假定这些数据可以自由再分发。
|
||||
|
||||
## 本项目自建部分
|
||||
|
||||
`scripts/` 下的整理脚本由本项目编写,依据 GPL-3.0 提供,与组织内其他仓库一致。
|
||||
@@ -0,0 +1,17 @@
|
||||
# 公共辅助码
|
||||
|
||||
供 Windows、Apple 和 Linux 共用的辅助码数据。
|
||||
|
||||
所有辅助码文件统一放在 `helpcodes\` 目录下。当前 server 使用的方案和文件为:
|
||||
|
||||
- `lantian`:`helpcode.txt`(蓝天小雨点)
|
||||
- `ziranma`:`zrm_helpcode_big_unique.txt`(自然码)
|
||||
- `shouyou2_0`:`shouyou2_0_helpcode.txt`(首右2.0)
|
||||
- `shouyouplus`:`shouyouplus_helpcode.txt`(首右plus)
|
||||
- `xiaohe`:`xiaohe_helpcode.txt`(小鹤)
|
||||
|
||||
公共数据位于 Engine 的 `helpcode/helpcodes/`,平台打包时复制需要的文件到应用资源目录。运行时路径由平台传给引擎,不依赖作者机器的绝对路径。合仓后的消费端无需再单独检出 HelpCode 仓库。
|
||||
|
||||
## 参考
|
||||
|
||||
- 自然码辅助码:<https://github.com/copperay/ZRM_Aux-code>
|
||||
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,61 @@
|
||||
"""筛选自然码辅助码文件中辅助码恰好为两位字符的记录。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
DEFAULT_INPUT = Path(
|
||||
r"C:\Users\SonnyCalcr\Downloads\ZRM_Aux-code-main\ZRM_Aux-code_4.3.txt"
|
||||
)
|
||||
DEFAULT_OUTPUT = Path(__file__).resolve().parent.parent / "helpcodes" / "zrm_helpcode.txt"
|
||||
|
||||
|
||||
def filter_helpcode(source: Path, destination: Path) -> tuple[int, int]:
|
||||
"""保留 `字=两位辅助码` 记录,按完整记录去重并维持原顺序。"""
|
||||
seen: set[str] = set()
|
||||
result: list[str] = []
|
||||
matched_count = 0
|
||||
|
||||
with source.open("r", encoding="utf-8-sig") as input_file:
|
||||
for raw_line in input_file:
|
||||
line = raw_line.rstrip("\r\n")
|
||||
if "=" not in line:
|
||||
continue
|
||||
|
||||
_, helpcode = line.split("=", 1)
|
||||
if len(helpcode) != 2:
|
||||
continue
|
||||
|
||||
matched_count += 1
|
||||
if line not in seen:
|
||||
seen.add(line)
|
||||
result.append(line)
|
||||
|
||||
destination.parent.mkdir(parents=True, exist_ok=True)
|
||||
with destination.open("w", encoding="utf-8", newline="\n") as output_file:
|
||||
output_file.write("\n".join(result))
|
||||
if result:
|
||||
output_file.write("\n")
|
||||
|
||||
return matched_count, len(result)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(
|
||||
description="筛选辅助码恰好为两位字符的记录,并按完整记录去重。"
|
||||
)
|
||||
parser.add_argument("input", nargs="?", type=Path, default=DEFAULT_INPUT)
|
||||
parser.add_argument("output", nargs="?", type=Path, default=DEFAULT_OUTPUT)
|
||||
args = parser.parse_args()
|
||||
|
||||
matched, written = filter_helpcode(args.input, args.output)
|
||||
print(f"读取文件:{args.input}")
|
||||
print(f"两位辅助码记录:{matched} 条")
|
||||
print(f"去重后写入:{written} 条")
|
||||
print(f"输出文件:{args.output}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,69 @@
|
||||
"""根据 helpcode.txt 中已有的汉字筛选 small 版自然码辅助码表。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
DEFAULT_INPUT = ROOT / "helpcodes" / "zrm_helpcode_big.txt"
|
||||
DEFAULT_REFERENCE = ROOT / "helpcodes" / "helpcode.txt"
|
||||
DEFAULT_OUTPUT = ROOT / "helpcodes" / "zrm_helpcode.txt"
|
||||
|
||||
|
||||
def read_keys(path: Path) -> set[str]:
|
||||
"""读取等号左侧的全部键(汉字或词条)。"""
|
||||
keys: set[str] = set()
|
||||
with path.open("r", encoding="utf-8-sig") as file:
|
||||
for raw_line in file:
|
||||
line = raw_line.rstrip("\r\n")
|
||||
if "=" in line:
|
||||
key, _ = line.split("=", 1)
|
||||
keys.add(key)
|
||||
return keys
|
||||
|
||||
|
||||
def create_small_table(source: Path, reference: Path, output: Path) -> tuple[int, int]:
|
||||
"""保留 source 中键存在于 reference 的所有记录,按完整行去重。"""
|
||||
wanted_keys = read_keys(reference)
|
||||
seen: set[str] = set()
|
||||
result: list[str] = []
|
||||
|
||||
with source.open("r", encoding="utf-8-sig") as file:
|
||||
for raw_line in file:
|
||||
line = raw_line.rstrip("\r\n")
|
||||
if "=" not in line:
|
||||
continue
|
||||
key, _ = line.split("=", 1)
|
||||
if key in wanted_keys and line not in seen:
|
||||
seen.add(line)
|
||||
result.append(line)
|
||||
|
||||
with output.open("w", encoding="utf-8", newline="\n") as file:
|
||||
file.write("\n".join(result))
|
||||
if result:
|
||||
file.write("\n")
|
||||
|
||||
return len(wanted_keys), len(result)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(
|
||||
description="按 helpcode.txt 已有汉字筛选 small 版辅助码表。"
|
||||
)
|
||||
parser.add_argument("input", nargs="?", type=Path, default=DEFAULT_INPUT)
|
||||
parser.add_argument("reference", nargs="?", type=Path, default=DEFAULT_REFERENCE)
|
||||
parser.add_argument("output", nargs="?", type=Path, default=DEFAULT_OUTPUT)
|
||||
args = parser.parse_args()
|
||||
|
||||
key_count, line_count = create_small_table(
|
||||
args.input, args.reference, args.output
|
||||
)
|
||||
print(f"参考汉字:{key_count} 个")
|
||||
print(f"写入记录:{line_count} 条")
|
||||
print(f"输出文件:{args.output}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,3 @@
|
||||
$target = Join-Path $env:LOCALAPPDATA 'metasequoiaime\helpcodes'
|
||||
New-Item -ItemType Directory -Path $target -Force | Out-Null
|
||||
Copy-Item -Path ".\helpcodes\*" -Destination $target -Recurse -Force
|
||||
Reference in new issue
Block a user