Word does deliver the bare-Shift release to the key-event sink; what it
does not deliver is a fresh GetKeyState() snapshot alongside it. Hosts
that call into ITfKeystrokeMgr outside the message dispatch see the
modifier state as of the last message the thread pulled, so
Global::PureShiftKeyUp/IsShiftKeyDownOnly -- both derived from
GetKeyState() -- can still describe the previous keystroke. The release
match then silently fails even though OnTestKeyUp and OnKeyUp both ran.
Match on the released virtual key plus _shiftHotkeyArmed instead.
The latch already means "this Shift went down alone and nothing else has
been pressed since" (_TrackModifierHotkeyArming disarms on any other
key-down) and it self-clears, so TestKeyUp/KeyUp cannot double-toggle.
Arming reads GetAsyncKeyState(), which reports the real keyboard rather
than the host thread's queue. Same treatment for bare Ctrl.
This is how Weasel and WindInput handle it -- neither installs a
keyboard hook for the toggle -- so revert the hook back to mintty-only.
mintty is a genuinely different failure: it offers no bare modifier
key-up to the sink at all, so there is no event to match against.
70930df stopped eating the bare-Shift release so JetBrains double-Shift
keeps working, and moved the toggle into OnTestKeyUp because TSF does not
call OnKeyUp after a FALSE test result. Word never completes a bare-Shift
release through ITfKeyEventSink at all, so neither callback runs and the
CN/EN toggle was lost there.
The mintty workaround already solves exactly this shape: a thread-local
WH_KEYBOARD hook watches the real bare-Shift press/release and feeds it
back into the deferred hotkey path, with _MarkBareShiftHandled() latching
presses the key-event sink already toggled so hosts that do deliver the
release never toggle twice. Install that hook for every host instead of
only mintty.exe, and rename its members off the mintty-specific naming.
Also log the three bare-Shift release paths under [issue47] -- key-up was
previously uninstrumented, which is why the Word diagnostic log could not
distinguish "release never arrived" from "toggle queued but dropped".
* chore: settle line endings in .gitattributes
Nothing declared line endings, so each file kept whatever the contributor's core.autocrlf produced when it was added: 382 text files ended up LF and 50 CRLF. The CRLF ones are the hazard. Editing one on macOS or Linux writes LF lines into it, and clang-format rejects a file that mixes both -- which is how `quality / Workflow validation` went red on KeyStateCategory.h, where three lines out of a CRLF file had arrived as LF. #309 converted those three lines back and left the other 49 files loaded.
The repository now declares LF and the working trees are renormalized to match, so the failure mode is gone rather than deferred to whichever of those files is edited next.
The four binary types are declared rather than left to git's heuristic, which decides from the first 8000 bytes alone. user_dict.dat is caught by that sample today only because the 40-byte seed is almost entirely NUL.
Verified: the 56 renormalized files are byte-identical to their old contents once CR is stripped, no text file in the index still carries CRLF, and scripts/format.sh --check passes.
* chore: skip the line-ending renormalization in blame
The commit before this one rewrote every line of 56 files without changing a character of them, which is what this file exists to hide.
Two host-timing paths could hide or displace the candidate window between
ordinary keystrokes. Both get much more likely while the machine is busy,
which is when the flicker is reported.
TS_E_NOLAYOUT from ITfContextView::GetTextExt means "not laid out yet, ask
again", but _GetTextExt reported it as a valid measurement at INVALID_Y.
Server then parks the host off-screen for that frame and the next successful
measurement pulls it back. Chromium-based hosts answer TS_E_NOLAYOUT
whenever their renderer is behind, so under load this repeats per keystroke.
Keep the anchor the session already has instead, once it has one; composition
start still falls back to the sentinel rather than reusing a stale anchor
from another document.
TF_LC_DESTROY was treated as candidate-session teardown for every process
except a hardcoded Telegram.exe. Hosts that transiently destroy and recreate
their context view mid-composition therefore got a HideCandidate between
keystrokes, which clears the live composition on the Server side: the window
blinks and the pending candidate can no longer be committed. Gate on a live
composition instead of a process name. Commit, cancel, focus loss, and
presenter cleanup all end the composition first, so they still tear the
session down through their own paths.
Elevated setup left high-IL config.toml that medium-IL Settings never
rewrote on machines hit by the ANSI path bug. Make the data directory
writable, migrate ACP-mangled leftovers, rebuild unparseable files, and
keep remaining asset paths on wchar_t.
Fixes#242
std::filesystem::path(std::string) and path::string() convert through the
system ANSI code page, so a non-ASCII (e.g. Chinese) user profile path on a
non-UTF-8 (GBK 936) system was corrupted and every config read/write failed
with "设置保存失败"; on a code page that cannot represent the characters
path::string() threw, crashing the Server at startup in a watchdog restart loop.
Never route config paths through the ANSI narrow encoding:
- build g_config_path and the legacy voice path from the wide accessors;
- read config by opening the wide path + toml::parse(text) instead of
toml::parse_file(path.string()) (LoadImeConfig, MigrateLegacyVoiceInputConfig);
- build and open the TSF shared config path as a wide std::filesystem::path.
Add a regression test that round-trips config read/write under a Chinese
profile path.
* fix(server): publish candidate pages as immutable snapshots
Global::candidate_ui is a plain global with no lock. The IPC worker rebuilds and destroys its vectors while the UI thread paints from them, and the handoff is an asynchronous PostMessage, so the worker returns to its loop and pops the next task without waiting. A second keystroke, or an async english.db candidate arrival, could free the page_views buffer and its strings while FillItemsFromUi held a reference into it. ShowItemContextMenu reads the same vectors from a mouse callback with no message handoff at all.
The worker now copies each finished page into an immutable snapshot and publishes it at the single point where the rebuild completes; every UI-thread reader loads that snapshot once and reads only its own copy. PostMessage is deliberately left alone: the worker also serves UI-originated tasks, so a synchronous handoff would invite a cross-thread deadlock.
* fix(server): bound the aux and diagnostic pipe reads
Both listeners serve a single pipe instance inline and polled a NOWAIT handle for as long as the process ran, breaking only on a read that succeeded or failed with something other than ERROR_NO_DATA. A client that connected and never wrote held the only instance forever, so the loop never returned to ConnectNamedPipe and no later client was accepted, permanently losing the langbar right-click channel and the terminal-deactivation teardown fallback.
Give both loops the same deadline the hello reads already use and go back to accepting when it expires.
* fix(server): snapshot the voice config and replace config.toml atomically
GetConfiguredVoiceInput handed out a reference to a global that LoadImeConfig rewrites from the IPC worker thread, clearing and refilling two std::map members and around twenty strings. The voice control thread and the low-level keyboard hook, which copies the struct on every keystroke, walked those nodes as they were freed. LoadImeConfig now parses into a local and publishes it in one step under a process-local shared_mutex, and the getter returns a snapshot by value. The file lock is deliberately not reused for readers: it is a named cross-process mutex with a five second wait, and taking it in a keyboard hook would stall typing system-wide.
WriteConfiguredValue also reopened the live config.toml with ios::trunc, so a crash mid-write destroyed the user settings and could leave the file unparseable for good. It now goes through WriteFileTextAtomically, which this file already uses elsewhere. That makes the directory entry swap atomic; it does not flush, so it is not a durability guarantee.
* fix(ipc): restrict the server pipes to the owning account
The pipe DACL granted Everyone FILE_ALL_ACCESS. That bit set includes FILE_CREATE_PIPE_INSTANCE, so any local account could add an instance to a name this server already owned and answer part of the connections in its place. The names are machine-global with no per-user component, and the TSF client opened them with no SQOS, so a squatter could also impersonate the connecting thread.
Build the descriptor from the process owner SID and admit only LocalSystem, that account, and app containers with connect-only rights so UWP text input still works; fail pipe creation outright when the SID cannot be resolved rather than falling back to something permissive. Pass FILE_FLAG_FIRST_PIPE_INSTANCE on the first instance of each name so a squatted name fails loudly, and request SECURITY_IDENTIFICATION on the client side.
Authenticating the peer process after connect is a larger change to the handshake and is not attempted here.
* fix(tsf): guard the null focus paths and unadvise the compartment sink
_CandidateChangeNotification treated ITfThreadMgr::GetFocus and ITfDocumentMgr::GetTop as if they could only fail through a FAILED HRESULT. Both return S_OK with a null out-pointer when nothing has focus or the document stack is empty, and the results were dereferenced unconditionally, so an external UILess host calling Finalize while focus moved away crashed the host application. The sibling _RequestCancelComposition in the same class already checks both.
CLangBarItemButton::CleanUp only unregistered the compartment inside the SUCCEEDED branch of a CoCreateInstance, then deleted the sink regardless. This TIP registers COM-less activation support, where that call does fail, and the sink was freed while TSF still held it advised. Unadvising never needed the thread manager, so it now runs unconditionally before the deletes.
* fix(ui): drop the cached visuals when the scene is replaced
Window caches focused, hovered and captured Visual pointers into the scene tree, and SetScene destroyed that tree without clearing them. Toggling the input mode rebuilds the floating toolbar scene, so the next mouse move dispatched a virtual call through a freed object.
Clear all three before the assignment, and release the capture only when this window actually holds it: an adopted HWND may never have called SetCapture, and capturedVisual_ is set even when it did not.
* fix(settings): scope external skin CSS through the CSS parser
injectScopedSkinCss concatenated a skin toolbar.css into an @scope block as text. Block nesting is textual, so a leading close brace in the skin ended the block early and the rest styled the whole settings window. The host validates only the skin file name, never the body, and candidatePreviewCss interpolated unvalidated colour strings into rule bodies the same way.
Create the scope rule empty and re-insert every rule parsed out of the skin as a child of it, so nothing lands outside however the braces are balanced, and accept a colour only in the plain notations we actually render.
* fix(release): gate the dictionary source commit on its default branch
verify_published iterated only data["repositories"], but the dictionary block pins a source_commit exactly like they do. The manifest check merely proves the release agrees with the lock about that commit, which a branch commit satisfies just as happily, so a dictionary built from an unmerged branch shipped with the provenance the gate exists to refuse.
Run the dictionary through the same loop under its own name, resolving its repository from the lock. A retired dictionary repository that no longer answers is reported rather than failing the release: its published artifacts are immutable history, and being unable to re-check them is not the same as finding them unmerged.
Windows, Server and UiHtml each carried their own gitlink to MSIME-Engine, all three pinned to the same commit, and Server carried a gitlink to MSIME-UI that is now the ui/ component. Git only reads the root .gitmodules, so the nested ones had to go regardless.
The engine and the two third-party dependencies now live once at vendor/MetasequoiaImeEngine, vendor/opencc and vendor/cpp-pinyin. Server reaches them through MSIME_VENDOR_DIR and builds ui/ as an out-of-tree subdirectory; the Windows tip's relative contract includes gained the extra level; the ui-html contract sync and the Server env/table generators follow the engine to its new location.
Preparation for folding Server, UI, UiHtml, Installer, ImeLog and TsfEditControl into this repository. The Windows tip's own sources, build files and local build scripts move under windows/ so every component sits at the same level. Product-level files stay at the root: the lock, the release manifest, the shared CI scripts, the integration tests, the docs and the Engine submodule.
Ouchihao