17 Commits
Author SHA1 Message Date
fanlusky 9f5572c525 fix(build): 所有 MSVC 目标显式按 UTF-8 编译源文件
#548 修了 ngram,同类问题还有几处:源文件或头文件是无 BOM 的 UTF-8,
目标却没有 /utf-8,在代码页 936 的系统上注释会吞掉换行。

- ui:Controls.h 让 msimeui 与各 demo 单独构建失败,/utf-8 设为 PUBLIC
- server/tests/portable:input_key_policy.h 让 MSVC 构建失败
- engine test_windows_ipc_contract、tests/pipe-probe、windows/tests:C4819
- windows DLL:此前只靠 fmt 的 INTERFACE 选项带来 /utf-8,改为显式设置,
  否则 L"“" 等宽字符标点会在去掉 fmt 后静默变成乱码
2026-09-30 15:37:13 +08:00
fanlusky a7779d2b05 refactor(engine): 将 engine 从 submodule 内嵌为仓内一等组件
`vendor/MetasequoiaImeEngine` submodule 改为 `engine/` 目录,导入的是
product-lock.json 原本钉住的提交 c810d201,逐文件核对过 blob,因此这是一次
搬迁而不是引擎升级。引擎自己的嵌套 submodule(googlepinyinime-rev、utfcpp、
miniaudio)按各自 gitlink 一并展开,来源与提交见 engine/UPSTREAM.md。

Windows 专门化裁掉了不属于本产品的部分:whisper.cpp 与本地推理 provider、
macOS 示例宿主、整个 dictionary/ 建库流水线、上游 CI 与 release-please 配置。
词库仍按 product-lock.json 在构建时取发布产物。

引擎不再进 product-lock.json:没有 gitlink 就没有可比对的东西,锁里写一个上游
commit 等于宣称这棵树仍等于上游,而第一个 Windows 专门化改动落地时这句话就假了。
verify-contracts / verify-checkout 随之移除,verify-published 保留。

CI 与工具链:引擎源码并入根 scripts/format.sh(删掉引擎自己的那份,避免两套排除
列表分叉);引擎 voice.yml 里的 silero_vad.onnx 摘要校验移到本仓 lock job;
engine-bump-triage 工作流与其测试删除;gitsubmodule 的 dependabot 频率由 daily
改为 monthly。

README、README.en、AGENTS.md、CONTRIBUTING、docs/product-release.md 及各子目录
文档同步说明新的位置与边界。
2026-09-12 03:40:48 +08:00
Aegon8719 6ff1cff42d issue-52 加入简繁切换快捷键;增加候选词翻页快捷键组合、增加以词定字快捷键组合;
issue-55 候选词与辅助码间保留一个半角空格。
2026-09-08 14:07:48 +08:00
Evan Hu 88720a80df fix(ci): 修复引擎 bump 自动合并的守卫,并补回 Issue 模板的脱敏警告
engine-bump-triage.yml 里 `&& steps.lock.outputs.locked == 'yes'` 被写进了 HEAD 这个环境变量的值,而不是步骤的 if:。它在那个位置根本不是条件:不论契约锁校验通过与否,自动合并都会执行;步骤随后发出的评论还会把 commit 后面跟上字面量 `&& steps.lock.outputs.locked == 'yes'` 一起打印出来。因为没有任何东西失败,这个问题没有暴露过。

(实际风险有限:Product inputs 是必需检查,未审的 bump 不会真的被放行。但这条守卫本身写了等于没写。)

补 tests/test_engine_bump_triage.py 三个断言:自动合并的条件必须同时包含判定和锁;任何步骤的 env 值里都不许再出现条件片段;评论用的 BASE/HEAD 必须只是 range 输出本身。第二条是通用的,同类错误以后写在别的步骤上也会被抓到。

同时补回本仓 bug_report.yml 相对组织模板丢掉的两项:日志字段的「不要粘贴真实输入内容、用户词库、API Key 或任何凭据」加粗警告,以及「这不是安全漏洞」的必选确认项。全组织的 Issue 流量几乎都在这个仓库,恰恰是最需要这两条的地方——输入法的日志天然贴近用户真实输入。
2026-09-06 18:15:51 +09:00
Evan ee4334dddd fix(ci): attach automatic release checks to the pull request (#175)
* ci(codeql): 避免版本元数据 PR 产生待人工批准的扫描

* fix(ci): attach release validation to the pull request
2026-09-06 11:10:05 +09:00
Evan 2562026a2a fix: 修复审计发现的候选页竞态、管道权限、TSF 空指针与皮肤 CSS 逃逸等八项缺陷 (#169)
* fix(server): publish candidate pages as immutable snapshots

Global::candidate_ui is a plain global with no lock. The IPC worker rebuilds and destroys its vectors while the UI thread paints from them, and the handoff is an asynchronous PostMessage, so the worker returns to its loop and pops the next task without waiting. A second keystroke, or an async english.db candidate arrival, could free the page_views buffer and its strings while FillItemsFromUi held a reference into it. ShowItemContextMenu reads the same vectors from a mouse callback with no message handoff at all.

The worker now copies each finished page into an immutable snapshot and publishes it at the single point where the rebuild completes; every UI-thread reader loads that snapshot once and reads only its own copy. PostMessage is deliberately left alone: the worker also serves UI-originated tasks, so a synchronous handoff would invite a cross-thread deadlock.

* fix(server): bound the aux and diagnostic pipe reads

Both listeners serve a single pipe instance inline and polled a NOWAIT handle for as long as the process ran, breaking only on a read that succeeded or failed with something other than ERROR_NO_DATA. A client that connected and never wrote held the only instance forever, so the loop never returned to ConnectNamedPipe and no later client was accepted, permanently losing the langbar right-click channel and the terminal-deactivation teardown fallback.

Give both loops the same deadline the hello reads already use and go back to accepting when it expires.

* fix(server): snapshot the voice config and replace config.toml atomically

GetConfiguredVoiceInput handed out a reference to a global that LoadImeConfig rewrites from the IPC worker thread, clearing and refilling two std::map members and around twenty strings. The voice control thread and the low-level keyboard hook, which copies the struct on every keystroke, walked those nodes as they were freed. LoadImeConfig now parses into a local and publishes it in one step under a process-local shared_mutex, and the getter returns a snapshot by value. The file lock is deliberately not reused for readers: it is a named cross-process mutex with a five second wait, and taking it in a keyboard hook would stall typing system-wide.

WriteConfiguredValue also reopened the live config.toml with ios::trunc, so a crash mid-write destroyed the user settings and could leave the file unparseable for good. It now goes through WriteFileTextAtomically, which this file already uses elsewhere. That makes the directory entry swap atomic; it does not flush, so it is not a durability guarantee.

* fix(ipc): restrict the server pipes to the owning account

The pipe DACL granted Everyone FILE_ALL_ACCESS. That bit set includes FILE_CREATE_PIPE_INSTANCE, so any local account could add an instance to a name this server already owned and answer part of the connections in its place. The names are machine-global with no per-user component, and the TSF client opened them with no SQOS, so a squatter could also impersonate the connecting thread.

Build the descriptor from the process owner SID and admit only LocalSystem, that account, and app containers with connect-only rights so UWP text input still works; fail pipe creation outright when the SID cannot be resolved rather than falling back to something permissive. Pass FILE_FLAG_FIRST_PIPE_INSTANCE on the first instance of each name so a squatted name fails loudly, and request SECURITY_IDENTIFICATION on the client side.

Authenticating the peer process after connect is a larger change to the handshake and is not attempted here.

* fix(tsf): guard the null focus paths and unadvise the compartment sink

_CandidateChangeNotification treated ITfThreadMgr::GetFocus and ITfDocumentMgr::GetTop as if they could only fail through a FAILED HRESULT. Both return S_OK with a null out-pointer when nothing has focus or the document stack is empty, and the results were dereferenced unconditionally, so an external UILess host calling Finalize while focus moved away crashed the host application. The sibling _RequestCancelComposition in the same class already checks both.

CLangBarItemButton::CleanUp only unregistered the compartment inside the SUCCEEDED branch of a CoCreateInstance, then deleted the sink regardless. This TIP registers COM-less activation support, where that call does fail, and the sink was freed while TSF still held it advised. Unadvising never needed the thread manager, so it now runs unconditionally before the deletes.

* fix(ui): drop the cached visuals when the scene is replaced

Window caches focused, hovered and captured Visual pointers into the scene tree, and SetScene destroyed that tree without clearing them. Toggling the input mode rebuilds the floating toolbar scene, so the next mouse move dispatched a virtual call through a freed object.

Clear all three before the assignment, and release the capture only when this window actually holds it: an adopted HWND may never have called SetCapture, and capturedVisual_ is set even when it did not.

* fix(settings): scope external skin CSS through the CSS parser

injectScopedSkinCss concatenated a skin toolbar.css into an @scope block as text. Block nesting is textual, so a leading close brace in the skin ended the block early and the rest styled the whole settings window. The host validates only the skin file name, never the body, and candidatePreviewCss interpolated unvalidated colour strings into rule bodies the same way.

Create the scope rule empty and re-insert every rule parsed out of the skin as a child of it, so nothing lands outside however the braces are balanced, and accept a colour only in the plain notations we actually render.

* fix(release): gate the dictionary source commit on its default branch

verify_published iterated only data["repositories"], but the dictionary block pins a source_commit exactly like they do. The manifest check merely proves the release agrees with the lock about that commit, which a branch commit satisfies just as happily, so a dictionary built from an unmerged branch shipped with the provenance the gate exists to refuse.

Run the dictionary through the same loop under its own name, resolving its repository from the lock. A retired dictionary repository that no longer answers is reported rather than failing the release: its published artifacts are immutable history, and being unable to re-check them is not the same as finding them unmerged.
2026-09-06 10:31:48 +09:00
Evan Hu 0baaf9a1d0 fix(server): 将本地测试入口接回合仓后的主工程 2026-09-06 09:40:36 +09:00
Evan Hu b2d736cd79 fix(build): finish local tooling migration to the monorepo 2026-09-06 09:02:13 +09:00
Evan Hu fc0c996e05 chore(dict): follow the dictionary release to its semver tag
The dictionary release was republished as dict-v1.0.0 so that dictionary tags stop using calendar versions. The release was renamed rather than rebuilt, so the assets are byte-identical and every SHA256 in product-lock.json is unchanged; only the tag string moves. Downloads under the old tag now return 404, so this pin has to follow.

The runbook example and the fixture standing in for a current release move to the semver form too. LEGACY_DICTIONARY_TAG and the retired-repository fixture keep their calendar names because they identify historical releases that stay immutable.
2026-09-06 08:47:38 +09:00
Evan Hu c3fcc4a515 refactor(product): consume the consolidated Engine data and shared voice capture
Ports metasequoiaime/MSIME-Windows#153 onto the consolidated repository. That pull request was written against the six-repository layout: it edited product-ci.yml, which no longer exists, and re-pinned Server, UiHtml and Installer, which are directories now.

Three of its pinned commits had not reached this repository at all. The consolidation took each source repository at its main, and Server, UiHtml and Installer each landed two more commits afterwards, so dropping the pins as merely obsolete would have silently dropped the work the pull request exists to consume. The two substantive ones are ported here: Server drops its vendored miniaudio.h and builds against the engine's MetasequoiaIme::VoiceCapture, and the installer collects helpcodes from the engine. UiHtml's was an engine gitlink bump, which one submodule resolves on its own.

The engine is pinned to d0dc0c2b rather than to the b7c799ef that pull request named. b7c799ef predates 403b4a4d, which is what this repository pins today and is the commit that added pagination to the settings dictionary contract, so taking the pull request's pin verbatim would have reverted a contract main already depends on and stripped offset, limit and hasMore back out of the generated web bindings. d0dc0c2b is the engine's main and is the same commit the dictionary release declares as its source, so the code and the data now name one commit.

The helpcodes moved into the engine, so MSIME-HelpCode stops being a product input: the gitlink that pins the contracts pins the tables too. product-lock.json is down to the engine and the dictionary release, and the dictionary itself now ships from MSIME-Engine. Its manifest states which commit built the data and whether that tree was clean, and the lock verifies both, so matching digests can no longer stand in for a source nobody can rebuild from. The retired MSIME-Dict repository stays valid for the one tag that shipped from it and nothing else.

The release job that packages now checks out submodules, because the helpcodes it collects live in one.
2026-09-06 04:13:04 +09:00
Evan Hu b48dc82652 fix(ci): resolve the pipe probe contracts from the repository, not the server
The probe took its include directory from $ServerRoot/MetasequoiaImeEngine/contracts, which is where the engine sat when it was a submodule of Server. It is a submodule of the repository now, so the path does not move with -ServerRoot. A missing include directory is not a configure error, so this surfaced minutes later as C1083 on a header nobody had moved; the script now checks and says so before building.

The local packaging rehearsals had the same stale assumption, looking for MetasequoiaImeTsf, MetasequoiaImeServer and MetasequoiaImeUiHtml beside the installer. Worse, they skipped the build when a neighbour was absent, which in this layout means every build is skipped and whatever binaries are lying around get packaged into an installer you then run on your own machine. The three components are directories of this repository, so a missing one is now a hard failure, and Prepare-PackageFiles.ps1 is passed the directory names it needs.

Both were missed because the sweep that moved the submodules read its own output through head, so a truncated list looked like a complete one. Re-checked by resolving every relative path literal in every tracked file with no filtering.
2026-09-06 03:28:13 +09:00
Evan Hu 32e722c176 ci: build every component from the one checkout
The product lock existed because five repositories had to be pinned to reproduce one product. Four of them are directories now, so a commit of this repository already pins them and there is nothing left for the lock to say about them. What it still records is what the product consumes from outside: the engine submodule, the helpcodes and the dictionary release.

CI gains a job per component instead of a reusable workflow that checked out locked commits and rebuilt them. The release workflow no longer stages five sibling checkouts under historical names and no longer carries a second sparse checkout for its own scripts; Prepare-PackageFiles.ps1 already took the source directory names as parameters, so it is told the new ones, plus where the product-wide notice now lives.

Two checks the removed cross-repository pins used to provide are replaced in kind: the pages are verified against the engine submodule's web contract in CI rather than by locking UiHtml and Engine together, and the engine bump triage that Server carried moves here and now also says so when a bump leaves the lock behind.
2026-09-06 02:35:49 +09:00
Evan Hu 1c6c956530 feat(product): lock the shared-session stack and validate dictionary formats 2026-09-06 00:53:26 +09:00
Evan Hu a81bc47c71 feat(product): require locked commits to be on their default branch at release
The lock pins immutable commits, but nothing checked that anybody merged them. Today it pins server 40d85a4 and engine 9a4b0c6, both of which exist only on architecture/product-contracts branches, and a release built from this commit would have shipped and attested to a Server that never reached main. The release job's existing provenance check covers only the TSF commit being released, not the inputs the lock names.

verify-published compares each locked commit against its own repository's default branch and accepts only behind or identical, the two states that mean it is an ancestor. ahead and diverged both mean the commit sits on something nobody merged.

It runs in the release path and deliberately not in product-ci.yml. A pull request legitimately locks branch commits while one change lands across several repositories at once, which is exactly the situation this branch is in; enforcing reachability in pull request CI would deadlock the landing it exists to protect. At release time the judgement inverts, because an input that never reached its default branch is an input nobody reviewed.
2026-09-05 23:18:03 +09:00
Evan Hu 9b2844534e feat(product): keep WebView and native contracts in one locked combination 2026-09-05 23:10:29 +09:00
Evan Hu 1c045f1738 test(ipc): exercise real Server handshakes from x86 and x64 clients 2026-09-05 22:41:45 +09:00
Evan Hu 487011c802 feat(product): lock release inputs and share negotiated IPC contracts 2026-09-05 22:32:21 +09:00