fanlusky
80b1fc42a1
词格原先的整句打分是一份没校准过的启发式 unigram，质量很差。这里内嵌 libime 在用的 kenlm 查询子集，改由它的三元模型 sc.lm 来打路径分。 - engine/ngram 只编 kenlm 的 query / trie-build 路径。不定义 WITH_THREADS， parallel_read.cc 因此退化成串行实现，整个子集零 Boost 依赖。 KENLM_MAX_ORDER=3 必须与建模阶数一致，改它等于读不懂已发布的模型文件。 - ngram::LanguageModel 是唯一 include kenlm 头文件的地方，对外只给出一个纯 std 的头，_HAS_AUTO_PTR_ETC 和 kenlm 的 include 目录都保持 PRIVATE。 - shared_language_model 按资源路径共享一份 mmap。kenlm 的查询是只读的且把 上下文放在调用方的 State 里，所以并发会话不需要额外加锁。 - 模型缺失时 valid() 为假，词格退回原来的启发式打分：整句候选只是变差，不会 停止工作。测试覆盖了缺模型降级与按路径共享。 - 出货的 ARPA 不带 <s>/</s>，所以解码从 null_state() 起算而非 begin_state()。 kenlm 是上游副本，不重排版、不进 clang-format，其 LGPL-2.1+ 与 BSD 例外清单 已补进 THIRD_PARTY_NOTICES.txt。googlepinyinime 那条 Fallback 整句路径不变。
Product contracts
These headers are independently consumable C++17 contracts. Including them does not link the input engine, start threads, access dictionaries, or require any third-party code. The TSF text service consumes only these headers; the server consumes the same source by linking the engine. Both live in this repository — windows/ and server/ — and so does the engine, so one commit covers all three and there is no pin to keep in step.
windows_ipc.h owns the wire layouts, pipe names and opcodes. voice_composition_pipe.h owns voice framing. Consumers may wrap these headers but must not duplicate their definitions.
punctuation/policy.json is the source of truth for the ASCII punctuation keys and their Chinese
translations. Its generated policy.h is consumed by the engine and included directly by the hosts
that route punctuation keys. Stateful quote alternation and nested book-title marks remain
part of the same contract, rather than being re-created in each host.
product_lock.py contains the shared product-lock primitives used by the three platform locks.
See product_lock.md for the vendoring and byte-equality rule.
Main handshake
Reverse endpoints retain their 16-byte hello and PipeReady acknowledgement. Once both endpoints are registered, the new client sends the existing 304-byte ClientHello with:
| Field | Meaning |
|---|---|
| keycode | MSIP magic |
| wch | Major wire protocol version |
| point[0] | Minor additive version |
| modifiers_down | Supported capability bits |
| point[1] | Required capability bits |
| request_id | Nonzero registration correlation ID |
The Server sends ProtocolReady or ProtocolMismatch on the registered reply endpoint before activating the main route. The reply echoes the correlation ID and contains major/minor, negotiated capabilities and magic in the first six UTF-16 units. These packets are consumed only during registration and are never text commits.
Existing DLLs with unversioned hello remain accepted by the new Server using the established v1 semantics; they receive no new opcode. New DLLs require an acknowledgement from the new Server within the existing bounded handshake budget. Against an old or incompatible Server they remain disconnected and use the existing raw-input fallback, rather than interpreting unnegotiated frames. A late acknowledgement from an old registration cannot authorize another one.
Append opcodes; never change released values or reuse them. An incompatible layout needs a new major protocol and an explicit migration, not another copy of a header. Minor additions must be optional capabilities.
CharacterSetShortcut is optional and is not included in the default capability set.
Windows implementations that support it pass Capabilities | CharacterSetShortcut
to Hello / Negotiate. Only after negotiating this bit may a Chinese-mode client
consume Ctrl+Shift+F (no Alt or Windows modifier) and send the existing KeyEvent
with keycode F and Shift/Control modifier bits. The server toggles the persisted
character set without clearing composition, and refreshes the current candidate
page and toolbar. It sends no key reply. Clients suppress auto-repeat and must not
replay an ambiguously delivered toggle; older peers simply omit the optional bit.
tests/windows_ipc_contract.cpp executes wire-layout/upgrade/framing cases on all Engine CI platforms and both Windows TSF architectures.
backend/protocol.json 定义 MSIME-Backend 的可选共通 HTTP 服务 API，详见后端协议与兼容性。