PrepareCandidateList runs on the shared task thread, so a stall inside it
delays the hide/show messages queued behind it and reaches the screen as
candidate-window flicker. The diagnostic log showed an identical workload
take 8 ms once and 688 ms moments later, but not which segment paid for it.
Split the function into measured segments and emit one "candidate build"
line per keystroke, separating the engine candidate lookup from the
user-dictionary fixed-position pass -- the latter opens a SQLite
connection and runs its schema DDL on every call.
Measurement only; no behaviour change.
The unconditional 50ms hide grace cost typing its snap: every commit left the
candidate window up for an extra frame or three, and when the next word started
inside the window the pending hide was cancelled outright, so the box stopped
disappearing between words and just morphed instead.
Only a hide that waited behind a stalled task thread can describe state the user
has already typed past, and only that one has a show queued behind it worth
waiting for. Measure the queue wait where the task is dispatched and mark the
posted message; normal typing never reaches the threshold, so it hides
immediately exactly as before.
The wait is now logged per hide task, so the threshold can be checked against a
real capture rather than assumed.
Diagnostic logs from a busy machine show the candidate host repainting far
more often than the input changes. Three separate causes, all in the UI
thread's handling of the worker's posted messages:
- A HideCandidate for one keystroke of a burst (backspacing the composition
empty, then typing again) cloaks a visible window and un-cloaks it ~30ms
later. Defer the hide on a short grace timer and drop it when a show
arrives first; a real commit or focus loss still hides, 50ms later.
- Each keystroke posts two WM_SHOW_MAIN_WINDOW messages, since the candidate
rebuild and the English/cloud merge both request a show. Skip a repaint
whose preedit, page, selection and caret match the frame already on screen
within a short window.
- When the worker stalls it drains several keystrokes at once and posts a
show for each. The handler always renders the newest published state, so an
older queued show can only repaint what the newer one is about to; drop it
when a newer show is already queued.
The stall itself (PrepareCandidateList blocking the shared task thread, worst
case 688ms against a median of 11ms) is untouched here.
The sub-page nav titles were English ("Sticker", "GIF", "Kaomoji",
"Symbols") while the Clipboard page was already Chinese, and the text
labels the tabs fall back to when no icon font carries the glyph are all
Chinese. On a Windows 10 machine that actually hits that fallback, the user
taps a tab labelled 贴纸 / 颜文字 / 符号 and lands on a page headed
"Sticker" / "Kaomoji" / "Symbols".
Translate the titles to match the tab labels and drop the per-page font and
size branch, which only existed because Clipboard was the one Chinese title.
Rename kClipboardTitleFontSize to kNavTitleFontSize now that every page uses
it, and move it next to the other nav metrics. "GIF" stays as is: it is an
acronym and the tab label already reads GIF.
The floating toolbar and the emoji panel hardcoded "Segoe Fluent Icons",
which ships with Windows 11 only. DirectWrite substitutes a font silently
instead of failing, so on Windows 10 every private-use codepoint the toolbar
draws (U+E982, U+F110, U+F138, ...) rendered as a blank box: the buttons
still worked but no longer showed any state.
Resolve the font per codepoint rather than per family, because "Segoe MDL2
Assets" is a subset of Segoe Fluent Icons and swapping the family alone would
still draw tofu wherever a glyph is missing:
- add msimeui::ResolveIconGlyph(), which probes the installed icon fonts with
IDWriteFont::HasCharacter() and returns the first family that really carries
the glyph, or no family at all;
- give every toolbar and tab icon a text label to fall back on (中/英, 半/全,
简/繁, ...) when neither font has the glyph;
- drop the unused ToolbarIconButton::SetGlyph/SetLabel, which would bypass the
resolved family.
Add unit tests for the resolver, covering the missing-glyph path that cannot
be reached on a host that has Segoe Fluent Icons installed.
std::filesystem::path(std::string) and path::string() convert through the
system ANSI code page, so a non-ASCII (e.g. Chinese) user profile path on a
non-UTF-8 (GBK 936) system was corrupted and every config read/write failed
with "设置保存失败"; on a code page that cannot represent the characters
path::string() threw, crashing the Server at startup in a watchdog restart loop.
Never route config paths through the ANSI narrow encoding:
- build g_config_path and the legacy voice path from the wide accessors;
- read config by opening the wide path + toml::parse(text) instead of
toml::parse_file(path.string()) (LoadImeConfig, MigrateLegacyVoiceInputConfig);
- build and open the TSF shared config path as a wide std::filesystem::path.
Add a regression test that round-trips config read/write under a Chinese
profile path.
* fix(server): publish candidate pages as immutable snapshots
Global::candidate_ui is a plain global with no lock. The IPC worker rebuilds and destroys its vectors while the UI thread paints from them, and the handoff is an asynchronous PostMessage, so the worker returns to its loop and pops the next task without waiting. A second keystroke, or an async english.db candidate arrival, could free the page_views buffer and its strings while FillItemsFromUi held a reference into it. ShowItemContextMenu reads the same vectors from a mouse callback with no message handoff at all.
The worker now copies each finished page into an immutable snapshot and publishes it at the single point where the rebuild completes; every UI-thread reader loads that snapshot once and reads only its own copy. PostMessage is deliberately left alone: the worker also serves UI-originated tasks, so a synchronous handoff would invite a cross-thread deadlock.
* fix(server): bound the aux and diagnostic pipe reads
Both listeners serve a single pipe instance inline and polled a NOWAIT handle for as long as the process ran, breaking only on a read that succeeded or failed with something other than ERROR_NO_DATA. A client that connected and never wrote held the only instance forever, so the loop never returned to ConnectNamedPipe and no later client was accepted, permanently losing the langbar right-click channel and the terminal-deactivation teardown fallback.
Give both loops the same deadline the hello reads already use and go back to accepting when it expires.
* fix(server): snapshot the voice config and replace config.toml atomically
GetConfiguredVoiceInput handed out a reference to a global that LoadImeConfig rewrites from the IPC worker thread, clearing and refilling two std::map members and around twenty strings. The voice control thread and the low-level keyboard hook, which copies the struct on every keystroke, walked those nodes as they were freed. LoadImeConfig now parses into a local and publishes it in one step under a process-local shared_mutex, and the getter returns a snapshot by value. The file lock is deliberately not reused for readers: it is a named cross-process mutex with a five second wait, and taking it in a keyboard hook would stall typing system-wide.
WriteConfiguredValue also reopened the live config.toml with ios::trunc, so a crash mid-write destroyed the user settings and could leave the file unparseable for good. It now goes through WriteFileTextAtomically, which this file already uses elsewhere. That makes the directory entry swap atomic; it does not flush, so it is not a durability guarantee.
* fix(ipc): restrict the server pipes to the owning account
The pipe DACL granted Everyone FILE_ALL_ACCESS. That bit set includes FILE_CREATE_PIPE_INSTANCE, so any local account could add an instance to a name this server already owned and answer part of the connections in its place. The names are machine-global with no per-user component, and the TSF client opened them with no SQOS, so a squatter could also impersonate the connecting thread.
Build the descriptor from the process owner SID and admit only LocalSystem, that account, and app containers with connect-only rights so UWP text input still works; fail pipe creation outright when the SID cannot be resolved rather than falling back to something permissive. Pass FILE_FLAG_FIRST_PIPE_INSTANCE on the first instance of each name so a squatted name fails loudly, and request SECURITY_IDENTIFICATION on the client side.
Authenticating the peer process after connect is a larger change to the handshake and is not attempted here.
* fix(tsf): guard the null focus paths and unadvise the compartment sink
_CandidateChangeNotification treated ITfThreadMgr::GetFocus and ITfDocumentMgr::GetTop as if they could only fail through a FAILED HRESULT. Both return S_OK with a null out-pointer when nothing has focus or the document stack is empty, and the results were dereferenced unconditionally, so an external UILess host calling Finalize while focus moved away crashed the host application. The sibling _RequestCancelComposition in the same class already checks both.
CLangBarItemButton::CleanUp only unregistered the compartment inside the SUCCEEDED branch of a CoCreateInstance, then deleted the sink regardless. This TIP registers COM-less activation support, where that call does fail, and the sink was freed while TSF still held it advised. Unadvising never needed the thread manager, so it now runs unconditionally before the deletes.
* fix(ui): drop the cached visuals when the scene is replaced
Window caches focused, hovered and captured Visual pointers into the scene tree, and SetScene destroyed that tree without clearing them. Toggling the input mode rebuilds the floating toolbar scene, so the next mouse move dispatched a virtual call through a freed object.
Clear all three before the assignment, and release the capture only when this window actually holds it: an adopted HWND may never have called SetCapture, and capturedVisual_ is set even when it did not.
* fix(settings): scope external skin CSS through the CSS parser
injectScopedSkinCss concatenated a skin toolbar.css into an @scope block as text. Block nesting is textual, so a leading close brace in the skin ended the block early and the rest styled the whole settings window. The host validates only the skin file name, never the body, and candidatePreviewCss interpolated unvalidated colour strings into rule bodies the same way.
Create the scope rule empty and re-insert every rule parsed out of the skin as a child of it, so nothing lands outside however the braces are balanced, and accept a colour only in the plain notations we actually render.
* fix(release): gate the dictionary source commit on its default branch
verify_published iterated only data["repositories"], but the dictionary block pins a source_commit exactly like they do. The manifest check merely proves the release agrees with the lock about that commit, which a branch commit satisfies just as happily, so a dictionary built from an unmerged branch shipped with the provenance the gate exists to refuse.
Run the dictionary through the same loop under its own name, resolving its repository from the lock. A retired dictionary repository that no longer answers is reported rather than failing the release: its published artifacts are immutable history, and being unable to re-check them is not the same as finding them unmerged.
Ports metasequoiaime/MSIME-Windows#153 onto the consolidated repository. That pull request was written against the six-repository layout: it edited product-ci.yml, which no longer exists, and re-pinned Server, UiHtml and Installer, which are directories now.
Three of its pinned commits had not reached this repository at all. The consolidation took each source repository at its main, and Server, UiHtml and Installer each landed two more commits afterwards, so dropping the pins as merely obsolete would have silently dropped the work the pull request exists to consume. The two substantive ones are ported here: Server drops its vendored miniaudio.h and builds against the engine's MetasequoiaIme::VoiceCapture, and the installer collects helpcodes from the engine. UiHtml's was an engine gitlink bump, which one submodule resolves on its own.
The engine is pinned to d0dc0c2b rather than to the b7c799ef that pull request named. b7c799ef predates 403b4a4d, which is what this repository pins today and is the commit that added pagination to the settings dictionary contract, so taking the pull request's pin verbatim would have reverted a contract main already depends on and stripped offset, limit and hasMore back out of the generated web bindings. d0dc0c2b is the engine's main and is the same commit the dictionary release declares as its source, so the code and the data now name one commit.
The helpcodes moved into the engine, so MSIME-HelpCode stops being a product input: the gitlink that pins the contracts pins the tables too. product-lock.json is down to the engine and the dictionary release, and the dictionary itself now ships from MSIME-Engine. Its manifest states which commit built the data and whether that tree was clean, and the lock verifies both, so matching digests can no longer stand in for a source nobody can rebuild from. The retired MSIME-Dict repository stays valid for the one tag that shipped from it and nothing else.
The release job that packages now checks out submodules, because the helpcodes it collects live in one.
The emoji, keyboard and handwriting panels embed the GUI framework's icon through a relative path that was written when msimeui was a submodule under server/. It now lives one level further out, at the repository root.
Found by the resource compiler rather than by the sweep that moved the submodules: that sweep matched CMake, JSON, PowerShell, Python, shell and YAML by extension, and .rc was not on the list. Re-checked by resolving every relative path literal in every tracked file regardless of extension; these three were the only ones left that did not resolve.
NormalizeClipboardText caps an entry at kMaxChars with text.resize(). wchar_t is UTF-16 on Windows, so that cap counts code units, and a cut landing between a high and low surrogate leaves a lone high surrogate. The result is not valid UTF-16, and it is what gets serialised into the history store and read back.
It takes a non-BMP character — an emoji, or a supplementary-plane Han character, both of which this IME can produce — sitting exactly at the boundary. The entry then carries an unpaired surrogate for as long as it stays in the history.
The cut now steps back one unit when it would split a pair, dropping the incomplete character instead of half of it. Entries that do not end on a surrogate are unaffected.
MSIME-Linux does not have this: its normalize() truncates with g_utf8_offset_to_pointer, which is codepoint aware. The two implementations otherwise agree exactly, including kMaxItems, kMaxChars and the promote-to-front policy.
src/d2d/candidate_window_d2d.{cpp,h} is 528 lines that no target compiles: it appears in no add_executable source list in CMakeLists.txt, and nothing outside the directory names candidate_window_d2d or CandidateWindowD2D. The d2d1 in CMakeLists.txt is the Windows system library, not this directory. It was last modified on 2025-06-25.
It is an earlier candidate window, superseded twice over: the shipped candidate window is the WebView2 page from MSIME-UiHtml, and the native GUI work now lives in the msimeui framework. Keeping a third, unbuilt implementation in the tree means every candidate window change invites the question of whether this one needs the same fix, and the answer is always no because it never runs.
The history keeps it if the Direct2D drawing is ever wanted as a reference.
The engine absorbed the six local mode queries during the cross-platform work,
and MSIME-Linux has been driving them in production since. This repository kept
its own implementations of the same six features, so date-time, emoji, kaomoji,
jianpin, quick-phrase and unicode each existed twice and were free to drift.
Deletes this repository's copies and points the eight call sites in
event_listener.cpp plus the two async workers at metasequoia::local_modes.
The shuangpin profile is the part to look at. The old modules resolved the
*configured* scheme themselves:
GetShuangpinProfile(shuangpin_schema.empty() ? GetConfiguredShuangpinSchema()
: shuangpin_schema)
The engine takes a resolved ShuangpinProfile and defaults it to Xiaohe. Taking
that default would have decoded J mode, emoji and kaomoji as Xiaohe for every
user on Ziranma, Shoudao or Microsoft shuangpin, so every call site passes the
configured profile explicitly through a ConfiguredShuangpinProfile() helper.
The database path needs the same care in the other direction. The old modules
took a UTF-8 std::string and handed it straight to sqlite3_open_v2; the engine
takes a std::filesystem::path and converts back with path_to_utf8 before
opening. The two async workers therefore bridge with path_from_utf8 rather than
constructing a path from the narrow string, which on Windows would interpret a
non-ASCII data directory in the active code page.
The four existing test files are kept and re-pointed at the engine functions
with every expectation byte-identical, including the seventeen date-time format
strings and the four shuangpin schemes jianpin covers. They are the equivalence
check: if the engine's port diverged from what this repository shipped, those
assertions fail rather than the difference reaching users.
Aegon
