Jia Sui
cc738f19ca
SentenceAssociationOptions 新增四个字段（模型路径、加性权重、重排开关、重排权重）， 随 QueryRequest 下发到全拼与双拼词典： - 加性（接入点 A）：collocation_scorer 在 Viterbi 逐边叠加字级搭配分，与三元模型 同域竞争，离线评测 top1 89.4% -> 95.6%（w=0.25） - 重排（接入点 B）：make_octagram_reranker 在解出的 n-best 上按整句搭配分重排， 以独立来源 CandidateSource::Collocation 出行（徽标〔万象〕），落库/学习判定与 神经整句同类 - 模型路径由宿主按模型包解析后传入，UTF-8 经 path_from_utf8 显式转换；模型缺席 时词格静默降级，与其余整句来源同范式 同时修正 test_pinyin 里一处 9 月底起与实现矛盾的断言：重排器同意词格排序时， 行保持词格来源（Generated）而非记到重排名下——以 word_lattice.h 的单重排器契约 与 merge 实现为准。原断言所在的 imetest 不进 CI，因此一直没被发现。
Input engine (engine/)
The C++ conversion engine behind Metasequoia IME for Windows: input sessions, candidate lookup, helpcodes, the shared contracts and the voice module.
This directory used to be a submodule of metasequoiaime/MSIME-Engine. It is now a component of this repository, a sibling of server/, ui/, windows/ and installer/. Its upstream commit, and exactly what the Windows specialization removed, are recorded in UPSTREAM.md. Treat it as first-party code: reviewed, formatted and tested like anything else here, no longer merged from upstream.
| Directory | Purpose |
|---|---|
core/, quanpin/, shuangpin/, english/, japanese/ |
Input sessions and candidate lookup |
contracts/ |
The authoritative IPC wire format, dictionary format, punctuation policy and WebView message schema |
helpcode/ |
Helpcode (形码) tables and generators |
local_modes/, providers/, user_dictionary/ |
Local modes, online candidate sources, the user dictionary |
voice/ |
Recording, WAV encoding, recognition and text cleanup; builds independently |
googlepinyinime-rev/, ngram/kenlm/, utfcpp/, voice/third_party/ |
Vendored third-party code, kept in its upstream formatting and licence |
Integration is through <metasequoia/session.h>. See runtime architecture.
Build
The product build does not build this directory on its own: server/CMakeLists.txt pulls it in with add_subdirectory, so CI compiles and exercises the engine through the server and the TSF frontend. After changing something here, run the server's build and tests.
Building it standalone still works and is much faster while working on the engine. Dependencies are CMake 3.25+, a C++17 compiler, Boost, fmt, spdlog and SQLite3.
vcpkg install --triplet x64-windows-static
cmake -S . -B build -DCMAKE_TOOLCHAIN_FILE="$env:VCPKG_INSTALLATION_ROOT/scripts/buildsystems/vcpkg.cmake" -DVCPKG_TARGET_TRIPLET=x64-windows-static
cmake --build build --config Release --parallel
ctest --test-dir build -C Release --output-on-failure --timeout 20
ctest runs every engine target registered in the root CMakeLists.txt: the segmentation, IPC and punctuation contracts, the platform API, data paths, runtime isolation, dictionary state and the per-mode session tests. tests/ additionally holds a separate Windows-only project that is not part of that build — add new tests to the registered targets so they actually run.
Formatting is driven from the repository root by scripts/format.sh, using the .clang-format in this directory (voice/ has its own with a wider column limit). The vendored third-party trees and the generated headers are excluded.
Contract rules
The IPC wire format, opcodes, voice framing and WebView message definitions live in contracts/ and are the single source of truth. Consumers include the same headers; do not restate them on either side. Dictionary naming is defined by contracts/dictionary/format.json, shared with the repository root's scripts/dictionary_product.py.
Data and licensing
The dictionaries are neither stored nor built here. They arrive as a release pinned by tag and digest in product-lock.json and are downloaded at build time; the pipeline that produced them left with the specialization. Per-source licensing is tracked in NOTICE.md — read it before redistributing the built databases.
Recognition goes through a cloud provider; local Whisper inference was removed. The only committed model is voice/assets/models/silero_vad.onnx, loaded solely by the standalone Windows host that is off by default.
Licence
GPL-3.0. Vendored googlepinyinime-rev, ngram/kenlm (LGPL-2.1-or-later), utfcpp and voice/third_party/miniaudio keep their upstream licences.