feat(server): 加入 S 模式符号输入与符号混输 (#1047)

符号数据改用 rime-wanxiang 的符号表与 typst/codex 符号名,源数据放在
msime-dictionary 的 sources/symbols/,替换原来的 piliapp 符号表。

- 建库:新增 symbols_db.py,生成表情面板用的 symbol_catalog(中文分类)和
  S 模式、符号混输用的 symbol_code;剔除 wanxiang-unrenderable.txt 列出的
  Windows 自带字体显示不了的符号,以及与 ① ❶ 看不出区别的 Dingbats 带圈数字
- 新增 scripts/check-symbol-rendering.py,用 Windows 字体生成上面的剔除清单
- Shift+S 进入符号模式,按万象分组码(xl、sx、szq)、数字读音(yi → 一 ① ⑴)
  或符号名(alpha、infinity)检索
- 符号混输:整串精确等于编码时混入,首个紧跟首选;读得成拼音的编码只混数字
  序号写法;双拼先按方案换算成全拼再查(小鹤 sj → san → ③)
- 表情面板符号页:中文分类,搜索把整词命中的符号列在「最佳匹配」
- 配置:utility.symbol_mode(默认开)、general.symbol_mixed_input(默认关)

Refs #795
This commit is contained in:
Lu Fan(SonnyCalcr) authored and GitHub committed 2026-10-18 02:08:39 +08:00
1 parent e9825a521f
commit 3cd429b19d
38 files changed
+936 -142

No files matched your search

+6 -4
View File
@@ -69,12 +69,12 @@ Windows 端的全部一方源码都在本仓,一次 clone 就能拿到完整
- 拼音方案支持辅助码(蓝天小雨点、自然码、首右 2.0、首右 Plus、小鹤、拼音加加),也可以导入自己的辅助码方案。
- 支持谷歌云候选、AI 联想(DeepSeek / OpenAI / SiliconFlow / Groq)。
- 竖排候选窗支持中英互译释义;本地未命中时可走腾讯云机器翻译。
- 候选混输(总开关 + 分项开关):英文、emoji、颜文字、快捷短语、日期时间;独立英文候选输入模式。
- 候选混输(总开关 + 分项开关):英文、emoji、颜文字、符号、快捷短语、日期时间;独立英文候选输入模式。
- 词库管理:全拼 / 五笔 / 英文查询、新增、批量导入与导出。
- 语音输入:豆包流式识别,以及 OpenAI / SiliconFlow / Groq 转写;可选文本润色。
- 手写识别、屏幕键盘、悬浮工具栏、剪贴板历史。
- 输入统计(默认关闭):只在本机记录字符数量与活跃时长,可在设置 → 统计中查看与清理。
- 快捷短语(K,可选混入候选)、日期时间(T)、U 模式(U,部件拆字反查)、Emoji(E)、颜文字(M)、超级简拼(J)、临时英文(Y)、临时日语(R)。
- 快捷短语(K,可选混入候选)、日期时间(T)、U 模式(U,部件拆字反查)、Emoji(E)、颜文字(M)、符号(S)、超级简拼(J)、临时英文(Y)、临时日语(R)。
- 智能标点(中文标点后按空格转英文标点,可撤回)、成对标点补全、以词定字、简繁转换。
- 多种皮肤:Fluent、微信绿、石墨 Graphite、杨柳青 Willow green、秋桂 Autumn osmanthus、微软 Microsoft,均支持深色 / 浅色。
@@ -224,14 +224,15 @@ Windows 端的全部一方源码都在本仓,一次 clone 就能拿到完整
### 候选混输和英文输入法模式
全拼 / 双拼输入中文时,可以把英文、emoji、颜文字、快捷短语和日期时间直接混进候选,不用切换到对应的快捷模式(五笔不混输)。这些开关集中在设置 → 实用功能的「候选混输」里:总开关关闭后全部不混入,展开后可逐项开关,各项的设置在总开关关闭期间保留。
全拼 / 双拼输入中文时,可以把英文、emoji、颜文字、符号、快捷短语和日期时间直接混进候选,不用切换到对应的快捷模式(五笔不混输)。这些开关集中在设置 → 实用功能的「候选混输」里:总开关关闭后全部不混入,展开后可逐项开关,各项的设置在总开关关闭期间保留。
- **中英混输**:根据字母串补充英文候选。触发字符数可选(1~8),达到该长度后才出现英文候选项。
- **emoji 混输**:匹配的 emoji 排在候选项列表最后,不挤占正常汉字候选的位置;**颜文字混输**排在末尾的 emoji 之后。翻页扩展候选时仍保留在列表末尾。
- **符号混输**(默认关闭):输入恰好等于某个符号编码时加入符号,例如 `alpha` → α、`infinity` → ∞、`szq` → ①…、`xl` → α β γ…。数字读音 `ling`、`yi`~`shi` 混入序号写法(`yi` → ① ⑴ ⒈)。双拼先按当前方案换算成全拼再查(小鹤 `sj` → `san` → ③),查不到再按原始按键查。只认整串精确匹配;其他读得成拼音的编码(`xi`、`ji`)只在 S 模式里用,不混进来。首个符号紧跟首个候选(打 `alpha` 第二位就是 α),其余排在颜文字之后。
- **快捷短语混入候选**:见下文「实用功能快捷模式」。
- **日期时间混入候选**(默认关闭):输入 `rq` / `riqi` / `date`、`sj` / `shijian` / `time`、`xq` / `xingqi` / `week` 时,当前日期、时间或星期出现在首个候选之后,其余格式排在列表末尾。双拼按当前方案换算后的整拼同样能触发,例如小鹤 `uijm` → `shijian`。
英文、emoji、颜文字和日期时间在同一条后台线程里查询,一次输入的结果一并合进候选,候选窗只刷新一次;快捷短语在组页时同步混入。
英文、emoji、颜文字、符号和日期时间在同一条后台线程里查询,一次输入的结果一并合进候选,候选窗只刷新一次;快捷短语在组页时同步混入。
在英文模式下,按下 `Ctrl + Shift + E` 可以切换到独立的英文输入法模式。这个模式与中英混输不同:中英混输是在拼音候选中补充英文,英文候选模式则专门按英文词库匹配当前字母串。
@@ -253,6 +254,7 @@ Windows 端的全部一方源码都在本仓,一次 clone 就能拿到完整
| U 模式(U) | `Shift + U` | 输入一个字各组成部分的拼音连写,候选出该字并反显读音。例如「晶」= 日+日+日,输入 `ririri`。空格上屏首选;数字选其他候选。 |
| Emoji(E) | `Shift + E` | 按全拼 / 简拼 / 双拼 / 英文关键词检索 emoji,例如 `xiaolian`、`xl`、`laugh`。 |
| 颜文字(M) | `Shift + M` | 按全拼 / 简拼 / 双拼 / 英文关键词检索颜文字,例如 `haixiu`、`kiss`。 |
| 符号(S) | `Shift + S` | 按分组码或符号名检索符号:`xl` / `xld` 希腊字母小写 / 大写、`sx` 数学符号、`szq` 带圈数字、`szh` 带括号数字、`lmd` 罗马数字、`jt` 箭头、`dw` 单位、`hb` 货币、`yi`~`shi` 数字 1~10 的各种写法;也可以打 typst 符号名,如 `alpha`、`infinity`、`arrowr`(arrow.r)。分组码来自 [rime-wanxiang](https://github.com/amzxyz/rime-wanxiang),Windows 自带字体显示不了的符号已剔除,看起来与 ① ❶ 一样的 Dingbats 带圈数字(➀ ➊)也不收。 |
| 超级简拼(J) | `Shift + J` | 之后每个字母都按简拼检索。全拼如 `nh` → 你好;双拼会按当前方案转换声母,例如小鹤 `nu` → `n'sh`(你说),与 `ns` → `n's` 不同。调频规则与全拼 / 双拼相同。 |
| 临时英文(Y) | `Shift + Y` | 之后字母按英文处理。空格上屏当前输入;数字可选英文词库单词。上屏后回到中文。 |
| 临时日语(R) | `Shift + R` | 之后字母按日语罗马字处理。上屏后回到中文。 |
+5
View File
@@ -1126,6 +1126,11 @@ Sources of msime.db, others.db and english.db:
Windows-classic-samples MIT https://github.com/microsoft/Windows-classic-samples
Google count_1w see source https://www.norvig.com/ngrams/count_1w.txt
OALD word forms none extracted headword list; rights with the dictionary's publisher
rime-wanxiang CC-BY-4.0 https://github.com/amzxyz/rime-wanxiang
typst/codex Apache-2.0 https://github.com/typst/codex
rime-prelude LGPL-3.0 https://github.com/rime/rime-prelude
The symbol tables in others.db (symbol_catalog and symbol_code, used by the emoji panel, S mode and mixed symbol candidates) are built from wanxiang_symbols.yaml and lua/data/codex_sym.txt of rime-wanxiang v18.0.16, copyright amzxyz and contributors, licensed under Creative Commons Attribution 4.0 International (https://creativecommons.org/licenses/by/4.0/). The data was changed: only the symbol groups are kept, the emoji, periodic table and XCompose sections are left out, symbols that Windows' bundled fonts cannot display are removed, and the groups are regrouped under Chinese headings and keyed for typing. wanxiang_symbols.yaml started as Rime's symbols.yaml from rime-prelude; codex_sym.txt was generated by rime-wanxiang from the symbol names of typst/codex.
The Chinese tables that msime.db is built from (cn/BaseDictAllV1Part1.txt and cn/BaseDictAllV1Part2.txt) merge rime-ice with CustomPinyinDictionary. rime-ice is licensed under the GNU General Public License version 3, which is the same license this project is distributed under, so its terms are satisfied by the distribution as a whole; this notice serves as its attribution, and its source remains available at the URL above.
+1
View File
@@ -53,6 +53,7 @@ set(METASEQUOIA_IME_ENGINE_SOURCES
local_modes/local_database.cpp
local_modes/quick_phrase_query.cpp
local_modes/radical_query.cpp
local_modes/symbol_query.cpp
local_modes/v_mode_query.cpp
neural/sentence_model.cpp
neural/neural_decoder.cpp
+3
View File
@@ -29,6 +29,9 @@ enum class CandidateSource
// 万象语法模型(octagram .gram)在词格 n-best 上重排挑中的那一句。句子仍是词格
// 解出来的,搭配强度表只负责挑;落库/学习判定与 NeuralDesktop/NeuralKeyboard 同类。
Collocation,
// Shift+S 符号模式与符号混输查出的符号(others.db 的 symbol_code)。与 emoji 一样直接上屏,
// 不进词库、不调频。
Symbol,
};
struct WordItem
@@ -134,6 +134,7 @@ bool counts_toward_quick_phrase_slot(const WordItem &item)
case CandidateSource::EnglishDictionary:
case CandidateSource::Emoji:
case CandidateSource::Kaomoji:
case CandidateSource::Symbol:
case CandidateSource::DateTime:
return false;
default:
+110
View File
@@ -0,0 +1,110 @@
#include "../contracts/assets/assets.h"
#include "symbol_query.h"
#include "local_database.h"
#include "../core/data_path.h"
#include <cstdint>
#include <memory>
#include <unordered_set>
namespace metasequoia::local_modes
{
namespace
{
struct StatementCloser
{
void operator()(sqlite3_stmt *statement) const
{
sqlite3_finalize(statement);
}
};
// 编码只有字母;统一转小写,非字母返回空串(查不到任何符号)。
std::string normalize_code(const std::string &code)
{
std::string lower;
lower.reserve(code.size());
for (const unsigned char character : code)
{
if (character >= 'a' && character <= 'z')
lower.push_back(static_cast<char>(character));
else if (character >= 'A' && character <= 'Z')
lower.push_back(static_cast<char>(character - 'A' + 'a'));
else
return {};
}
return lower;
}
LocalQueryResult query_failure(const char *diagnostic)
{
return {{}, std::string(diagnostic)};
}
// ?1 = 编码,?2 = 前缀上界,?3 = limit。返回 symbol 一列,同一符号只取排位最好的那行。
LocalQueryResult run_query(const std::string &code, const std::filesystem::path &database_path, const char *sql,
int limit)
{
const std::string lower = normalize_code(code);
if (lower.empty() || limit <= 0)
return {};
if (database_path.empty())
return query_failure("Symbol database is unavailable.");
const std::shared_ptr<sqlite3> database = open_local_database(database_path);
if (!database)
return query_failure("Symbol database is unavailable.");
sqlite3_stmt *raw_statement = nullptr;
if (sqlite3_prepare_v2(database.get(), sql, -1, &raw_statement, nullptr) != SQLITE_OK)
return query_failure("Symbol database could not be queried.");
const std::unique_ptr<sqlite3_stmt, StatementCloser> statement(raw_statement);
std::string upper_bound = lower;
upper_bound.push_back(static_cast<char>(0x7f));
// 前缀查询的同一符号可能挂在好几个编码下(a 与 ao、codex 的 alpha 与 Alpha),多取一些再去重。
const int fetch_limit = limit <= (1 << 20) ? limit * 4 : limit;
if (sqlite3_bind_text(statement.get(), 1, lower.c_str(), -1, SQLITE_TRANSIENT) != SQLITE_OK ||
sqlite3_bind_text(statement.get(), 2, upper_bound.c_str(), -1, SQLITE_TRANSIENT) != SQLITE_OK ||
sqlite3_bind_int(statement.get(), 3, fetch_limit) != SQLITE_OK)
return query_failure("Symbol database could not be queried.");
LocalQueryResult result;
std::unordered_set<std::string> seen;
int step_result = SQLITE_ROW;
while ((step_result = sqlite3_step(statement.get())) == SQLITE_ROW &&
result.candidates.size() < static_cast<std::size_t>(limit))
{
const auto *symbol = reinterpret_cast<const char *>(sqlite3_column_text(statement.get(), 0));
if (symbol == nullptr || !seen.insert(symbol).second)
continue;
result.candidates.emplace_back(lower, symbol, 0, CandidateSource::Symbol);
}
if (step_result != SQLITE_ROW && step_result != SQLITE_DONE)
return query_failure("Symbol database could not be queried.");
const std::size_t count = result.candidates.size();
for (std::size_t index = 0; index < count; ++index)
result.candidates[index].weight = static_cast<std::int64_t>(count - index);
return result;
}
} // namespace
LocalQueryResult query_symbols(const std::string &code, int limit)
{
return query_symbols(code, data_file_path(metasequoia::assets::other_dictionary), limit);
}
LocalQueryResult query_symbols(const std::string &code, const std::filesystem::path &database_path, int limit)
{
// 整串命中的编码排最前;其余前缀命中按编码长短,短的通常是这一类的总表(x 先出 xl 再出 xlc)。
constexpr const char *kSql = "SELECT symbol FROM symbol_code WHERE code>=?1 AND code<?2 "
"ORDER BY code IS NOT ?1,length(code),sort_order LIMIT ?3";
return run_query(code, database_path, kSql, limit);
}
LocalQueryResult query_mixed_symbols(const std::string &code, const std::filesystem::path &database_path, int limit)
{
constexpr const char *kSql = "SELECT symbol FROM symbol_code WHERE code=?1 AND code<?2 AND mixed=1 "
"ORDER BY sort_order LIMIT ?3";
return run_query(code, database_path, kSql, limit);
}
} // namespace metasequoia::local_modes
+18
View File
@@ -0,0 +1,18 @@
#pragma once
#include "local_query_result.h"
#include <filesystem>
#include <string>
namespace metasequoia::local_modes
{
// Shift+S 符号模式:按 others.db 的 symbol_code 查编码(万象分组码 xl、sx、szq,或 typst/codex 的符号名
// alpha、arrowr),整串等于编码的排最前,其余按编码前缀、短编码在前。编码只认字母,不分大小写。
LocalQueryResult query_symbols(const std::string &code, int limit = 200);
LocalQueryResult query_symbols(const std::string &code, const std::filesystem::path &database_path, int limit = 200);
// 符号混输:只认整串精确等于编码、且建库时标了可混输的行(编码读不成拼音,或数字读音下的 ① ⑴ ⒈)。
LocalQueryResult query_mixed_symbols(const std::string &code, const std::filesystem::path &database_path,
int limit = 3);
} // namespace metasequoia::local_modes
+44
View File
@@ -3,6 +3,7 @@
#include "../../local_modes/kaomoji_query.h"
#include "../../local_modes/quick_phrase_query.h"
#include "../../local_modes/radical_query.h"
#include "../../local_modes/symbol_query.h"
#include "../../local_modes/v_mode_query.h"
#include "../../core/data_path.h"
@@ -366,6 +367,16 @@ int run_test()
database.execute("INSERT INTO kaomoji VALUES('baifo','','longer key',140)");
// 单音节条目的简拼只有一个字母:打 k 时它精确命中 jianpin,但不该因此排到全拼前缀命中前面。
database.execute("INSERT INTO kaomoji VALUES('ku','k','jianpin initial',60)");
database.execute("CREATE TABLE symbol_code(code TEXT,symbol TEXT,sort_order INTEGER,mixed INTEGER)");
database.execute("INSERT INTO symbol_code VALUES('xl','α',10,1)");
database.execute("INSERT INTO symbol_code VALUES('xl','β',11,1)");
// 更长的编码目录位置更靠前,用来验证整串命中和短编码排在前面。
database.execute("INSERT INTO symbol_code VALUES('xlc','𝛂',5,1)");
database.execute("INSERT INTO symbol_code VALUES('xld','Α',12,1)");
database.execute("INSERT INTO symbol_code VALUES('xld','α',13,1)");
database.execute("INSERT INTO symbol_code VALUES('alpha','α',100,1)");
database.execute("INSERT INTO symbol_code VALUES('yi','一',20,0)");
database.execute("INSERT INTO symbol_code VALUES('yi','①',21,0)");
}
const auto emoji = metasequoia::local_modes::query_emoji("XIAOLIAN", SchemeType::Quanpin, others_database, 10);
@@ -448,6 +459,39 @@ int run_test()
kaomoji_initial.candidates[2].word == "jianpin initial",
"Kaomoji lookup let a jianpin exact match outrank full-pinyin prefix matches.");
// S 模式:整串命中的编码在前,其余前缀命中按编码长短;同一符号只出现一次。
const auto symbols = metasequoia::local_modes::query_symbols("XL", others_database, 200);
require(!symbols.diagnostic.has_value() && symbols.candidates.size() == 4 && symbols.candidates[0].word == "α" &&
symbols.candidates[1].word == "β" && symbols.candidates[2].word == "𝛂" &&
symbols.candidates[3].word == "Α" && symbols.candidates[0].pinyin == "xl" &&
symbols.candidates[0].source == CandidateSource::Symbol,
"Symbol lookup did not normalize, order, or deduplicate code matches.");
require(metasequoia::local_modes::query_symbols("xl", others_database, 1).candidates.size() == 1 &&
metasequoia::local_modes::query_symbols("", others_database, 10).candidates.empty() &&
metasequoia::local_modes::query_symbols("x'l", others_database, 10).candidates.empty() &&
metasequoia::local_modes::query_symbols("yi", others_database, 10).candidates.size() == 2,
"Symbol lookup did not honor its validation and limit rules.");
// 混输只认整串精确命中、且标了可混输的行:xl 不带出 xlc,没标可混输的行不混。
const auto mixed_symbols = metasequoia::local_modes::query_mixed_symbols("xl", others_database, 3);
require(mixed_symbols.candidates.size() == 2 && mixed_symbols.candidates[0].word == "α" &&
mixed_symbols.candidates[1].word == "β",
"Mixed symbol lookup matched more than the exact code.");
require(metasequoia::local_modes::query_mixed_symbols("yi", others_database, 3).candidates.empty() &&
metasequoia::local_modes::query_mixed_symbols("alph", others_database, 3).candidates.empty() &&
metasequoia::local_modes::query_mixed_symbols("alpha", others_database, 3).candidates.front().word ==
"α",
"Mixed symbol lookup did not respect the mixable flag and exact matching.");
// 旧词库没有 symbol_code 表:返回空列表和不带输入的诊断,不能崩。
const std::filesystem::path others_without_symbols = quick_phrase_directory / "private-others-old.db";
{
Database database(others_without_symbols);
database.execute("CREATE TABLE emoji_pinyin(key TEXT,emoji TEXT,sort_order INTEGER)");
}
const auto old_dictionary = metasequoia::local_modes::query_symbols("privatecode", others_without_symbols, 10);
require(old_dictionary.candidates.empty() && old_dictionary.diagnostic.has_value() &&
old_dictionary.diagnostic->find("privatecode") == std::string::npos,
"A dictionary without the symbol table lacked a privacy-safe diagnostic.");
const auto missing_emoji = metasequoia::local_modes::query_emoji(
"privatecode", SchemeType::Quanpin, quick_phrase_directory / "private-others-missing.db", 10);
require(missing_emoji.candidates.empty() && missing_emoji.diagnostic.has_value() &&
+5 -2
View File
@@ -64,6 +64,8 @@ candidate_translation_show_pos = true
emoji_mixed_input = false
# 中文输入状态下显示颜文字前缀候选(排在 emoji 之后)
kaomoji_mixed_input = false
# 中文输入状态下输入恰好等于符号编码(alpha、infinity、szq、xl,以及数字读音 yi → ① ⑴ ⒈)时显示符号候选,首个紧跟首选,其余排在颜文字之后
symbol_mixed_input = false
# 云候选
cloud_candidates = true
# TODO: 极简模式。极简模式下,候选窗口使用 d2d 绘制,界面只作展示候选项,无其他功能。
@@ -533,8 +535,8 @@ quick_phrase = true # Shift+K 进入快捷短语模式
quick_phrase_candidates = false # 全拼/双拼下把编码精确匹配的快捷短语混进候选,排在固定位置候选之后的首位
quick_phrase_frequency = true # 快捷短语参与调频:常越过它选别的候选时整组后退,最多退到首页末位
# 候选混输总开关:关闭后英文(general.cn_en_mixed_input)、emoji(general.emoji_mixed_input)、
# 颜文字(general.kaomoji_mixed_input)、快捷短语(quick_phrase_candidates)、日期时间(date_time_candidates)
# 都不再混进候选,各自的开关值保留
# 颜文字(general.kaomoji_mixed_input)、符号(general.symbol_mixed_input)、快捷短语(quick_phrase_candidates)、
# 日期时间(date_time_candidates)都不再混进候选,各自的开关值保留
mixed_candidates = true
date_time_candidates = false # 全拼/双拼下输入 rq / riqi / date、sj / shijian / time、xq / xingqi / week 时,把当前日期、时间、星期混进候选
date_time_menu = true # 日期时间混入候选时,在第三位放一个「📅日期」入口,选中后候选框换成全部日期格式(与 Shift+T 模式相同)
@@ -542,6 +544,7 @@ radical_mode = true # Shift+U 进入 U 模式(部件拆字反查):输入
date_time_mode = true # Shift+T 进入日期与时间快捷输入模式(全拼/双拼/五笔)
emoji_mode = false # Shift+E 进入 Emoji 快捷输入模式(支持全拼/简拼/双拼/英文);默认关闭,避免截走大写 E 开头的英文
kaomoji_mode = false # Shift+M 进入颜文字快捷输入模式(支持全拼/简拼/双拼/英文)
symbol_mode = true # Shift+S 进入符号快捷输入模式:输入分组码(xl 希腊字母、sx 数学、szq 带圈数字、jt 箭头)或符号名(alpha、infinity)
jianpin_mode = true # Shift+J 进入超级简拼模式(每个字母作为简拼;双拼按方案转换 zh/ch/sh)
y_mode = true # Shift+Y 进入临时英文模式(中文状态下直接打英文,上屏后回到中文)
r_mode = true # Shift+R 进入临时日语模式(中文状态下使用罗马字输入日语,上屏后回到中文)
+2 -2
View File
@@ -2,11 +2,11 @@
"""从 msime-dictionary 的文本源数据本地构建 Windows 端词库,写到 MetasequoiaImeDict/out/。
读取 msime-dictionary checkout 的 sources/ 与 custom/,加上 scripts/dictionary_build/inputs.lock.json 固定的
仓外文件(msime 的快捷短语、读音纠错、emoji/颜文字/符号表,ECDICT,SCOWL,rime-jp_sela),产出:
仓外文件(msime 的快捷短语、读音纠错、emoji/颜文字表,ECDICT,SCOWL,rime-jp_sela),产出:
msime.db 全拼表(含地名补充、自定义词条、读音纠错)、86 五笔、快捷短语、japanese_lexicon
english.db 英文候选、中英双向释义(ECDICT)、sources/cn-en/ 的中译英释义(全部义项)、custom/translations.txt 覆盖
others.db emoji、颜文字、符号
others.db emoji、颜文字、符号(sources/symbols/ 的万象符号表,剔除 wanxiang-unrenderable.txt 列出的)
dict_japanese.dat 日文整句模型(sources/japanese/ 的 Mozc 词库)
以及 Mozc 与 SCOWL 的声明、dictionary-manifest.json、SHA256SUMS.txt、local-dictionary.json
+144
View File
@@ -0,0 +1,144 @@
#!/usr/bin/env python3
"""用 Windows 自带字体检查 msime-dictionary 的符号源,把显示不了的符号写进 sources/symbols/wanxiang-unrenderable.txt。
build-dictionary.py 构建 others.db 时按这份清单剔除符号,构建本身不读字体,结果与机器无关;
换了符号源或想按新的 Windows 字体重新判定时,在 Windows 上重跑本脚本并评审清单的变化。
判定规则(逐条,命中第一条即记下):
invisible 只有格式字符、空白或组合附加符号,上屏后看不见或浮在前一个字上
private-use 含私用区码位,显示什么取决于碰巧装了哪种字体
no-glyph C:\\Windows\\Fonts 下(不算 Noto 系列这类非系统自带的字体)没有任何字体有它的字形;
emoji 序列改用 Segoe UI Emoji 排版判定
split Segoe UI Emoji 不能把序列合成一个字形(国旗、部分 ZWJ 组合),会显示成几个字符
需要 fonttools 与 uharfbuzz:python -m pip install fonttools uharfbuzz
用法:
python scripts/check-symbol-rendering.py # 默认读写与本仓同级的 msime-dictionary
python scripts/check-symbol-rendering.py --dictionary D:/msime-dictionary
"""
from __future__ import annotations
import argparse
import os
import platform
import subprocess
import sys
import unicodedata
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
from dictionary_build import build, symbols_db, textutil # noqa: E402
FONTS = Path(os.environ.get("WINDIR", r"C:\Windows")) / "Fonts"
# 不随 Windows 安装的字体:只出现在装过它们的机器上,不能拿来证明「Windows 能显示」。
NON_SYSTEM_PREFIXES = ("noto",)
IGNORABLE = {0x200D, 0xFE0E, 0xFE0F}
EMOJI_MARKERS = {0x200D, 0xFE0F, 0x20E3}
def load_fonts():
try:
import uharfbuzz as hb
from fontTools.ttLib import TTCollection, TTFont
except ImportError as error:
raise SystemExit("需要 fonttools 与 uharfbuzz:python -m pip install fonttools uharfbuzz") from error
coverage: set[int] = set()
used = 0
for path in sorted(FONTS.iterdir()):
suffix = path.suffix.lower()
if suffix not in (".ttf", ".otf", ".ttc") or path.name.lower().startswith(NON_SYSTEM_PREFIXES):
continue
try:
fonts = TTCollection(path, lazy=True).fonts if suffix == ".ttc" else [TTFont(path, lazy=True)]
for font in fonts:
coverage.update((font.getBestCmap() or {}).keys())
used += 1
except Exception: # noqa: BLE001 - 坏字体文件跳过,不影响其余判定
continue
emoji_path = FONTS / "seguiemj.ttf"
emoji_ttf = TTFont(emoji_path)
emoji_font = hb.Font(hb.Face(hb.Blob.from_file_path(str(emoji_path))))
version = emoji_ttf["head"].fontRevision
return hb, coverage, used, emoji_ttf.getBestCmap(), emoji_font, f"{version:.2f}"
def classify(symbol: str, hb, coverage: set[int], emoji_cmap: dict, emoji_font) -> str | None:
categories = [unicodedata.category(c) for c in symbol]
if all(cat in ("Cf", "Zs", "Zl", "Zp", "Cc", "Mn", "Me") for cat in categories):
return "invisible"
if "Co" in categories:
return "private-use"
codepoints = [ord(c) for c in symbol]
emoji = any(cp in EMOJI_MARKERS or 0x1F1E6 <= cp <= 0x1F1FF for cp in codepoints) or (
any(cp >= 0x1F000 for cp in codepoints) and all(cp in emoji_cmap or cp in IGNORABLE for cp in codepoints))
if emoji:
buffer = hb.Buffer()
buffer.add_str(symbol)
buffer.guess_segment_properties()
hb.shape(emoji_font, buffer, {})
if any(info.codepoint == 0 for info in buffer.glyph_infos):
return "no-glyph"
visible = sum(1 for position in buffer.glyph_positions if position.x_advance > 0)
return "split" if visible > 1 else None
if any(cp not in coverage and cp not in IGNORABLE for cp in codepoints):
return "no-glyph"
return None
def git_head(path: Path) -> str:
try:
commit = subprocess.run(["git", "-C", str(path), "rev-parse", "HEAD"], capture_output=True, text=True,
check=True).stdout.strip()
dirty = subprocess.run(["git", "-C", str(path), "status", "--porcelain", "--", "scripts"], capture_output=True,
text=True, check=True).stdout.strip()
except (OSError, subprocess.CalledProcessError):
return "unknown"
return commit + ("-dirty" if dirty else "")
def main() -> None:
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument("--dictionary", type=Path, default=build.ROOT.parent / "msime-dictionary")
args = parser.parse_args()
dictionary = args.dictionary.resolve()
groups = symbols_db.parse_wanxiang(textutil.read(dictionary / symbols_db.WANXIANG))
codex = symbols_db.parse_codex(textutil.read(dictionary / symbols_db.CODEX))
sources: dict[str, list[str]] = {}
for group in groups:
for symbol in group.items:
sources.setdefault(symbol, []).append(f"/{group.code}")
for name, symbol in codex:
sources.setdefault(symbol, []).append(f"codex:{name}")
hb, coverage, used, emoji_cmap, emoji_font, emoji_version = load_fonts()
lines = []
counts: dict[str, int] = {}
for symbol, where in sources.items():
reason = classify(symbol, hb, coverage, emoji_cmap, emoji_font)
if reason is None:
continue
counts[reason] = counts.get(reason, 0) + 1
codepoints = " ".join(f"U+{ord(c):04X}" for c in symbol)
shown = symbol if reason != "invisible" else ""
lines.append(f"{codepoints}\t{shown}\t{reason}\t{' '.join(dict.fromkeys(where))}")
header = [
f"# 由 MSIME-Windows 仓库提交 {git_head(build.ROOT)} 的 scripts/check-symbol-rendering.py 生成,不要手工编辑。",
f"# 判定环境:Windows {platform.version()},{used} 个 C:\\Windows\\Fonts 字体(不含 Noto 系列),Segoe UI Emoji {emoji_version}。",
f"# 检查对象:{symbols_db.WANXIANG} 中入选的符号分组与 {symbols_db.CODEX} 的全部符号,共 {len(sources)} 个不同的符号。",
"# 判定规则见生成脚本的说明:invisible 看不见,private-use 私用区,no-glyph 系统字体没有字形,split 序列不能合成一个字形。",
"# build-dictionary.py 构建 others.db 时剔除这里列出的全部符号,它们不进入表情面板,也不进入 S 模式和符号混输。",
f"# 共 {len(lines)} 个:" + ",".join(f"{name} {count}" for name, count in sorted(counts.items())),
"# 格式:码位<TAB>符号(invisible 留空)<TAB>原因<TAB>出处(万象分组编码或 codex:名称)",
]
target = dictionary / symbols_db.UNRENDERABLE
target.write_bytes(("\n".join(header + lines) + "\n").encode("utf-8"))
print(f"{len(lines)} 个显示不了的符号({counts})写入 {target}")
if __name__ == "__main__":
main()
+9 -5
View File
@@ -12,7 +12,7 @@ from contextlib import closing
from datetime import datetime, timezone
from pathlib import Path
from . import english_db, inputs, japanese_model, others_db, pinyin_db, textutil
from . import english_db, inputs, japanese_model, others_db, pinyin_db, symbols_db, textutil
ROOT = Path(__file__).resolve().parents[2]
DICT_ROOT = ROOT / "MetasequoiaImeDict"
@@ -155,10 +155,14 @@ def build_others(source: inputs.Inputs, path: Path, stage: Stages) -> None:
stage("kaomoji", kaomoji)
def symbols():
categories = others_db.parse_piliapp(textutil.read(source.resource("symbols/piliapp_symbols.txt")))
rows = others_db.symbol_rows(pinyin, categories)
others_db.build_symbols(connection, rows)
return f"{len(rows)} symbols"
groups = symbols_db.parse_wanxiang(textutil.read(source.source(symbols_db.WANXIANG)))
codex = symbols_db.parse_codex(textutil.read(source.source(symbols_db.CODEX)))
excluded = symbols_db.parse_unrenderable(textutil.read(source.source(symbols_db.UNRENDERABLE)))
catalog = symbols_db.catalog_rows(pinyin, groups, codex, excluded)
codes = symbols_db.code_rows(groups, codex, excluded, pinyin.syllables())
symbols_db.build(connection, catalog, codes)
return (f"{len(catalog)} panel symbols, {len(codes)} code rows from {len(groups)} groups and "
f"{len(codex)} codex names, {len(excluded)} unrenderable left out")
stage("symbols", symbols)
finally:
+1 -2
View File
@@ -10,8 +10,7 @@
"emoji/emoji.txt": {"size": 90178, "sha256": "a6a0af0c00e6e359eb2f1be247d15a5735c2240bc248181b6946e0377e991002"},
"emoji/emoji_catalog.txt": {"size": 47198, "sha256": "71b4027b02b457a2cea8b988cda907763e6ee8e241a8c9c353829a464f291d60"},
"emoji/emoji_en.txt": {"size": 183514, "sha256": "c1b0a92e5cfc6d23a972e4afac1770b1df5e8bc848ef397d2f785161e04cac48"},
"kaomoji/kaomoji.txt": {"size": 55524, "sha256": "0bfcd97533596496bb13652243931a4db45d47edc9f0e8241c722a98d84950ae"},
"symbols/piliapp_symbols.txt": {"size": 12790, "sha256": "37b6855f778ef01e112af4af68a8fe47caa6aee78b4b2b9eb39db1ae193d7fe8"}
"kaomoji/kaomoji.txt": {"size": 55524, "sha256": "0bfcd97533596496bb13652243931a4db45d47edc9f0e8241c722a98d84950ae"}
}
},
"external": {
+2 -2
View File
@@ -1,7 +1,7 @@
"""构建输入:msime-dictionary checkout,以及 inputs.lock.json 按大小和 SHA-256 固定、首次使用时下载并缓存的仓外文件。
仓外文件有三类:msime 仓库 resources/dictionary-sources/ 下人工维护的快捷短语、读音纠错、emoji、颜文字和符号表
(固定在 Release 所用的 msime 提交);ECDICT 与 SCOWL Aspell 包(Rust 构建器的锁文件同样固定它们);
仓外文件有三类:msime 仓库 resources/dictionary-sources/ 下人工维护的快捷短语、读音纠错、emoji 和颜文字
(固定在 Release 所用的 msime 提交;符号表改在 msime-dictionary 的 sources/symbols/);ECDICT 与 SCOWL Aspell 包(Rust 构建器的锁文件同样固定它们);
rime-jp_sela 的 jp_sela.dict.yaml(旧 msime.db 的 japanese_lexicon 来源)。
"""
+7 -95
View File
@@ -1,13 +1,11 @@
"""others.db:表情面板与 E/M 模式读的 emoji、颜文字和符号目录。
"""others.db:表情面板与 E/M 模式读的 emoji 和颜文字(符号在 symbols_db.py)。
对应 Rust 构建器 others.rs / pinyin.rs 的 emoji / kaomoji / symbols 阶段。关键词的拼音逐字取:
对应 Rust 构建器 others.rs / pinyin.rs 的 emoji / kaomoji 阶段。关键词的拼音逐字取:
Rust 用 pinyin crate,这里用 pypinyin 的单字表(同出 pinyin-data),多音字短语读音由 pinyin-overrides.txt 指定。
"""
from __future__ import annotations
import html
import re
import sqlite3
from . import textutil
@@ -66,6 +64,11 @@ class Pinyin:
items.append(other)
return items
def syllables(self) -> set[str]:
"""单字表里出现过的全部无声调音节(ü 写成 v)。"""
return {self._plain(reading).replace("ü", "v") for readings in self._readings.values()
for reading in readings.split(",")}
def full_and_initials(self, keyword: str) -> tuple[str, str]:
items = self.lazy(keyword)
return "".join(items), "".join(item[0] for item in items if item)
@@ -246,94 +249,3 @@ def build_kaomoji(connection: sqlite3.Connection, pinyin: Pinyin, mapping: dict[
[(kaomoji, order, " ".join(keywords)) for order, (kaomoji, keywords) in enumerate(mapping.items())])
textutil.analyze(connection, optimize=False)
return rows, len(mapping)
# ---- 符号 ----
_CATEGORY_LINE = re.compile(r"^# \[piliapp/symbol/([^/\]]+)/\]\s*(.+)\s*$")
# 父标签按导航顺序;每个 PiliApp 分类 slug 及其英文小标题,按小标题顺序。
PARENT_TABS = (
("Stars and shapes", (("star", "Stars"), ("asterisk", "Asterisks"), ("circle", "Circles"), ("triangle", "Triangles"),
("square", "Squares"), ("other-shapes", "Other shapes"), ("bullet-point", "Bullet points"))),
("Arrows and lines", (("arrow", "Arrows"), ("line", "Lines"), ("random-lines", "Random lines"))),
("Punctuation", (("punctuation", "Punctuation"), ("brackets", "Brackets"), ("quotation-mark", "Quotation marks"),
("pilcrow", "Pilcrow"), ("tick", "Check marks"), ("x-mark", "X marks"))),
("Math", (("math", "Math"), ("number", "Numbers"), ("fraction", "Fractions"), ("pi", "Pi"),
("subscript-superscript", "Super/subscripts"))),
("Currency", (("currency", "Currency"), ("business", "Business"), ("unit", "Units"))),
("Hearts", (("heart", "Hearts"), ("eye", "Eyes"), ("monochrome", "Monochrome"))),
("Letters", (("latin", "Latin letters"), ("latin-extended", "Latin extended"), ("greek", "Greek letters"),
("kana", "Japanese kana"), ("braille", "Braille"), ("korean", "Korean"))),
("Games", (("card-suit", "Card suits"), ("chess", "Chess"), ("dice", "Dice"))),
("Culture", (("culture", "Religion and culture"), ("cross", "Crosses"), ("zodiac", "Zodiac"), ("gender", "Gender"),
("totem", "Totems"))),
("Animals and nature", (("animals", "Animals"), ("flower", "Flowers"), ("weather", "Weather"))),
("People and activity", (("activity", "People and activity"),)),
("More", (("music", "Music"), ("tech", "Technical"), ("menu", "Menu"), ("misc", "Miscellaneous"),
("confidential", "Block elements"), ("mashup", "Mashup"))),
)
def parse_piliapp(text: str) -> dict[str, tuple[str, list[str]]]:
"""slug -> (中文标题, 符号)。解码 &dollar; 这类 HTML 实体;同一分类里重复的符号只留一个。"""
categories: dict[str, tuple[str, list[str]]] = {}
current = None
for raw in text.splitlines():
line = raw.strip()
if not line:
continue
match = _CATEGORY_LINE.match(line)
if match:
if current:
categories[current[0]] = (current[1], current[2])
current = (match.group(1).strip(), match.group(2).strip(), [], set())
continue
if line.startswith("#") or current is None:
continue
symbol = html.unescape(line).strip()
if symbol and symbol not in current[3]:
current[3].add(symbol)
current[2].append(symbol)
if current:
categories[current[0]] = (current[1], current[2])
return categories
def symbol_rows(pinyin: Pinyin, categories: dict[str, tuple[str, list[str]]]) -> list[tuple]:
rows = []
used = set()
for parent, slugs in PARENT_TABS:
for slug, category in slugs:
used.add(slug)
chinese, symbols = categories.get(slug, ("", []))
if not symbols:
continue
parts = [category, chinese, slug.replace("-", " "), parent]
if _is_cjk(chinese):
full, initials = pinyin.full_and_initials(chinese)
full, initials = full.lower(), initials.lower()
if full:
parts.append(full)
if initials and initials != full:
parts.append(initials)
keywords = " ".join(part for part in parts if part)
for symbol in symbols:
rows.append((symbol, category, parent, len(rows), keywords))
leftovers = [slug for slug, (_, symbols) in categories.items() if slug not in used and symbols]
if leftovers:
raise ValueError(f"unmapped PiliApp categories: {leftovers}")
return rows
def build_symbols(connection: sqlite3.Connection, rows: list[tuple]) -> None:
with connection:
connection.execute("DROP TABLE IF EXISTS symbol_catalog")
connection.execute("\n CREATE TABLE symbol_catalog (\n symbol TEXT NOT NULL,\n"
" category TEXT NOT NULL,\n parent_category TEXT NOT NULL,\n"
" sort_order INTEGER NOT NULL,\n keywords TEXT NOT NULL,\n"
" PRIMARY KEY (symbol, category)\n ) WITHOUT ROWID\n ")
connection.execute("CREATE INDEX idx_symbol_catalog_parent_order ON symbol_catalog(parent_category, sort_order)")
connection.executemany("INSERT INTO symbol_catalog (symbol, category, parent_category, sort_order, keywords) "
"VALUES (?, ?, ?, ?, ?)", rows)
textutil.analyze(connection, optimize=False)
+288
View File
@@ -0,0 +1,288 @@
"""others.db 的符号部分:表情面板的 symbol_catalog 与 S 模式、符号混输查的 symbol_code。
源数据在 msime-dictionary 的 sources/symbols/:
wanxiang_symbols.yaml rime-wanxiang 的符号表,取 symbols: 下 `'/编码': [符号, ...]` 分组
codex_sym.txt rime-wanxiang 从 typst/codex 生成的 `名称<TAB>符号`
wanxiang-unrenderable.txt Windows 自带字体显示不了的符号(scripts/check-symbol-rendering.py 生成),一律不入库
万象表里不是符号的分组不收:Emoji 分类(E 模式和表情面板已有 emoji)、元素周期表(元素信息文本)、
XCompose 合成(编码含标点,S 模式只能输入字母)。
"""
from __future__ import annotations
import re
import sqlite3
from dataclasses import dataclass
from . import textutil
from .others_db import Pinyin
WANXIANG = "sources/symbols/wanxiang_symbols.yaml"
CODEX = "sources/symbols/codex_sym.txt"
UNRENDERABLE = "sources/symbols/wanxiang-unrenderable.txt"
_GROUP_LINE = re.compile(r"""^ {4}(['"])/(.*?)\1:\s*\[(.*)\]\s*(?:#.*)?$""")
_COMMENT_LINE = re.compile(r"^ {4}#\s*(.*?)\s*$")
_CODE = re.compile(r"[A-Za-z]+\Z")
_CODEPOINT = re.compile(r"U\+([0-9A-F]{4,6})\Z")
# 万象用数字作编码的「同一个数的各种写法」:S 模式只能输入字母,改用读音作编码。
DIGIT_CODES = {"0": "ling", "1": "yi", "2": "er", "3": "san", "4": "si", "5": "wu", "6": "liu", "7": "qi", "8": "ba",
"9": "jiu", "10": "shi"}
# Dingbats 区的无衬线带圈数字 ➀–➉、➊–➓ 和 ①–⑩、❶–❿ 在候选窗里看不出区别,万象两套都收了,
# 列在一起就像同一个符号出现了两次,所以不收。
_VISUAL_DUPLICATES = ((0x2780, 0x2793),)
# 数字别名混进普通候选时只取序号写法:带圈 ①–⑩、带括号 ⑴–⑽、带点 ⒈–⒑、带圈零 ⓪。
_SEQUENCE_NUMBERS = ((0x2460, 0x2469), (0x2474, 0x247D), (0x2488, 0x2491), (0x24EA, 0x24EA))
# 万象的 /a…/z 里也收了数学字母(𝐚 𝑎 𝒂…),面板的「拉丁扩展」不重复列出,它们在「数学字母」标签页。
_MATH_ALPHANUMERIC = (0x1D400, 0x1D7FF)
_LATIN = tuple(code for letter in "abcdefghijklmnopqrstuvwxyz" for code in (letter, letter.upper()))
_MATH_STYLES = (("粗体", "sxc"), ("斜体", "sxx"), ("粗斜体", "sxz"), ("花体", "sxh"), ("花体粗体", "sxhc"),
("哥特体", "sxg"), ("哥特粗体", "sxgc"), ("双线体", "sxs"), ("无衬线体", "sxw"), ("无衬线粗体", "sxwc"),
("无衬线斜体", "sxwx"), ("无衬线粗斜体", "sxwz"), ("等宽体", "sxd"), ("希腊粗体", "xlc"),
("希腊斜体", "xlx"), ("希腊粗斜体", "xlz"), ("希腊无衬线粗体", "xlwc"), ("希腊无衬线粗斜体", "xlwz"))
# 表情面板符号页:父标签(面板上显示为图标,取第一个符号)→ 小标题 → 万象分组编码。没列出的分组只供 S 模式输入。
PANEL_TABS = (
("数学", (("数学符号", ("sx",)), ("大于小于", ("dy", "xy")), ("因为所以", ("yw", "sy")), ("上标", ("sb",)),
("下标", ("xb",)))),
("序号", (("带圈数字", ("szq",)), ("带括号数字", ("szh",)), ("带点数字", ("szd",)), ("罗马数字", ("lmd", "lm")),
("带圈字母", ("zmq",)), ("带括号字母", ("zmh",)), ("带圈汉字", ("hzq",)), ("带括号汉字", ("hzh",)),
("苏州码", ("szm",)), ("编号", ("num",)))),
("字母", (("希腊字母", ("xl", "xld")), ("俄文字母", ("ey", "eyd")), ("拼音", ("py", "pyd")), ("声调", ("sd",)),
("注音符号", ("zy",)), ("英文音标", ("yb",)), ("拉丁扩展", _LATIN))),
("数学字母", tuple((name, (code,)) for name, code in _MATH_STYLES)),
("单位", (("单位", ("dw",)), ("货币", ("hb",)), ("月份", ("yf",)), ("日期", ("rf",)), ("时间", ("sj",)))),
("箭头与图形", (("箭头", ("jt",)), ("几何图形", ("jh",)), ("方块", ("fk",)), ("五角星", ("wjx",)), ("星号", ("xh",)),
("圆点", ("yd",)), ("叉号", ("ih",)))),
("标点", (("点号", ("dd",)), ("非夹注符号", ("fj",)), ("夹注符号", ("jz",)), ("行间符号", ("hj",)),
("常用标点", ("bd",)), ("竖排标点", ("bdz",)), ("中英标点", ("bdzy",)))),
("杂项", (("常用符号", ("fh", "tsfh")), ("电脑按键", ("dn", "aj")), ("象棋", ("xq",)), ("麻将", ("mj",)),
("骰子", ("sz",)), ("扑克", ("pk",)), ("音乐", ("yy",)), ("性别", ("lx",)), ("天体", ("tt",)),
("星座", ("xz",)), ("六十四卦", ("lssg",)), ("太玄经", ("txj",)))),
("汉字部件", (("汉字结构", ("jg",)), ("偏旁", ("pp",)), ("康熙部首", ("kx",)), ("笔画", ("bh",)))),
("假名与韩文", (("平假名", ("jm",)), ("片假名", ("pjm",)), ("带圈假名", ("jmq",)), ("半角假名", ("jmbj",)),
("韩文字母", ("hw",)), ("带圈韩文", ("hwq",)), ("带括号韩文", ("hwh",)))),
)
@dataclass
class Group:
code: str
items: list[str]
def _flow_items(text: str, where: str) -> list[str]:
"""YAML 行内序列 `a, 'b', "c"`:单引号里 '' 表示一个单引号,双引号里认 \\" 与 \\\\。"""
items: list[str] = []
index, length = 0, len(text)
while index < length:
while index < length and text[index] == " ":
index += 1
if index >= length:
break
quote = text[index]
if quote in "'\"":
index += 1
value = []
while True:
if index >= length:
raise ValueError(f"{where}: unterminated quote")
char = text[index]
if quote == "'" and char == "'":
if index + 1 < length and text[index + 1] == "'":
value.append("'")
index += 2
continue
index += 1
break
if quote == '"' and char == "\\" and index + 1 < length:
value.append(text[index + 1])
index += 2
continue
if quote == '"' and char == '"':
index += 1
break
value.append(char)
index += 1
items.append("".join(value))
while index < length and text[index] == " ":
index += 1
if index < length and text[index] != ",":
raise ValueError(f"{where}: expected ',' after quoted item")
index += 1
else:
end = text.find(",", index)
end = length if end < 0 else end
value = text[index:end].strip()
if value:
items.append(value)
index = end + 1
return items
def parse_wanxiang(text: str) -> list[Group]:
"""symbols: 下的符号分组,按文件顺序。分段注释(Emoji 分类、元素周期表、XCompose)决定哪些分组不收。"""
lines = textutil.universal_lines(textutil.without_bom(text))
try:
start = lines.index("symbols:") + 1
except ValueError as error:
raise ValueError(f"{WANXIANG}: no top-level symbols: section") from error
groups: list[Group] = []
section = ""
for number, line in enumerate(lines[start:], start=start + 1):
match = _COMMENT_LINE.match(line)
if match:
for marker, name in (("Emoji 分类", "emoji"), ("元素周期表", "elements"), ("XCompose", "xcompose")):
if match.group(1).startswith(marker):
section = name
continue
match = _GROUP_LINE.match(line)
if not match:
continue
code = match.group(2)
items = [item for item in _flow_items(match.group(3), f"{WANXIANG}:{number}") if not _visual_duplicate(item)]
if not items or section in ("elements", "xcompose") or (section == "emoji" and code.startswith("e")):
continue
groups.append(Group(code, items))
return groups
def _visual_duplicate(symbol: str) -> bool:
return any(low <= ord(char) <= high for char in symbol for low, high in _VISUAL_DUPLICATES)
def parse_codex(text: str) -> list[tuple[str, str]]:
"""`名称<TAB>符号`;名称是 typst 的点分路径,如 arrow.r.double。"""
entries = []
for number, line in enumerate(textutil.universal_lines(textutil.without_bom(text)), start=1):
if not line or line.startswith("#"):
continue
name, separator, symbol = line.partition("\t")
if not separator or not name or not symbol:
raise ValueError(f"{CODEX}:{number}: expected name<TAB>symbol")
if not _visual_duplicate(symbol):
entries.append((name, symbol))
return entries
def parse_unrenderable(text: str) -> set[str]:
"""首列是空格分隔的码位(U+1F1E8 U+1F1F3),因为不可见字符没法直接写在行里。"""
symbols = set()
for number, line in enumerate(textutil.universal_lines(textutil.without_bom(text)), start=1):
if not line or line.startswith("#"):
continue
codepoints = line.split("\t", 1)[0].split(" ")
try:
symbols.add("".join(chr(int(_CODEPOINT.match(cp).group(1), 16)) for cp in codepoints))
except AttributeError as error:
raise ValueError(f"{UNRENDERABLE}:{number}: expected U+XXXX code points") from error
return symbols
def input_code(code: str) -> str | None:
"""S 模式能输入的编码:只含字母,统一小写(万象的 /a 与 /A 合成一个 a);数字编码改用读音。"""
if _CODE.match(code):
return code.lower()
return DIGIT_CODES.get(code)
def reads_as_pinyin(code: str, syllables: set[str]) -> bool:
"""整串能切成完整的拼音音节(yi、xi、zeta = ze + ta)。"""
reachable = [True] + [False] * len(code)
for end in range(1, len(code) + 1):
reachable[end] = any(reachable[start] and code[start:end] in syllables for start in range(max(0, end - 6), end))
return reachable[-1]
def code_rows(groups: list[Group], codex: list[tuple[str, str]], excluded: set[str],
syllables: set[str]) -> list[tuple]:
"""symbol_code 的行:(编码, 符号, sort_order, 可混输)。万象分组在前、按文件顺序,codex 名称在后。
可混输:符号混输只在普通拼音候选里认整串精确匹配。读得成拼音的编码(希腊字母名 xi、假名 ji)一混进来,
打「西」「几」都会冒出符号,所以只留给 S 模式。数字别名(yi、er)例外:打 yi 想要 ① 是常见用法,
但只混序号写法(① ⑴ ⒈ ⓪),一、壹、¹ 这些仍只在 S 模式里出。
"""
rows, seen = [], set()
def add(code: str, symbol: str, digit: bool = False) -> None:
if symbol in excluded or (code, symbol) in seen:
return
seen.add((code, symbol))
if digit:
mixed = 1 if len(symbol) == 1 and any(low <= ord(symbol) <= high for low, high in _SEQUENCE_NUMBERS) else 0
else:
mixed = 1 if len(code) >= 2 and not reads_as_pinyin(code, syllables) else 0
rows.append((code, symbol, len(rows), mixed))
for group in groups:
code = input_code(group.code)
if code is None:
continue
for symbol in group.items:
add(code, symbol, digit=group.code in DIGIT_CODES)
for name, symbol in codex:
code = name.replace(".", "").lower()
if _CODE.match(code):
add(code, symbol)
return rows
def catalog_rows(pinyin: Pinyin, groups: list[Group], codex: list[tuple[str, str]], excluded: set[str]) -> list[tuple]:
"""symbol_catalog 的行:(符号, 小标题, 父标签, sort_order, 关键词),同一父标签、同一小标题的行连续。
关键词给面板搜索用:小标题、父标签和它们的全拼与首字母,加上这个符号所在的全部万象编码与 codex 名称。
"""
by_code = {group.code: group for group in groups}
search: dict[str, list[str]] = {}
for group in groups:
code = input_code(group.code)
for symbol in group.items:
if code:
search.setdefault(symbol, []).append(code)
for name, symbol in codex:
search.setdefault(symbol, []).append(name)
def spelled(text: str) -> list[str]:
full, initials = pinyin.full_and_initials(text)
return [full, initials] if initials != full else [full]
rows = []
for parent, headings in PANEL_TABS:
for heading, codes in headings:
missing = [code for code in codes if code not in by_code]
if missing:
raise ValueError(f"{WANXIANG}: panel heading {heading!r} names missing groups {missing}")
base = [heading, *spelled(heading), parent, *spelled(parent)]
seen = set()
for code in codes:
for symbol in by_code[code].items:
if symbol in excluded or symbol in seen:
continue
if heading == "拉丁扩展" and any(_MATH_ALPHANUMERIC[0] <= ord(c) <= _MATH_ALPHANUMERIC[1] for c in symbol):
continue
seen.add(symbol)
keywords = " ".join(dict.fromkeys([*base, *search.get(symbol, [])]))
rows.append((symbol, heading, parent, len(rows), keywords))
return rows
def build(connection: sqlite3.Connection, catalog: list[tuple], codes: list[tuple]) -> None:
with connection:
connection.execute("DROP TABLE IF EXISTS symbol_catalog")
connection.execute("\n CREATE TABLE symbol_catalog (\n symbol TEXT NOT NULL,\n"
" category TEXT NOT NULL,\n parent_category TEXT NOT NULL,\n"
" sort_order INTEGER NOT NULL,\n keywords TEXT NOT NULL,\n"
" PRIMARY KEY (symbol, category)\n ) WITHOUT ROWID\n ")
connection.execute("CREATE INDEX idx_symbol_catalog_parent_order ON symbol_catalog(parent_category, sort_order)")
connection.executemany("INSERT INTO symbol_catalog (symbol, category, parent_category, sort_order, keywords) "
"VALUES (?, ?, ?, ?, ?)", catalog)
connection.execute("DROP TABLE IF EXISTS symbol_code")
connection.execute("\n CREATE TABLE symbol_code (\n code TEXT NOT NULL,\n"
" symbol TEXT NOT NULL,\n sort_order INTEGER NOT NULL,\n"
" mixed INTEGER NOT NULL,\n"
" PRIMARY KEY (code, symbol)\n ) WITHOUT ROWID\n ")
connection.executemany("INSERT INTO symbol_code (code, symbol, sort_order, mixed) VALUES (?, ?, ?, ?)", codes)
textutil.analyze(connection, optimize=False)
+5 -2
View File
@@ -65,6 +65,8 @@ candidate_translation_show_pos = true
emoji_mixed_input = false
# 中文输入状态下显示颜文字前缀候选(排在 emoji 之后)
kaomoji_mixed_input = false
# 中文输入状态下输入恰好等于符号编码(alpha、infinity、szq、xl,以及数字读音 yi → ① ⑴ ⒈)时显示符号候选,首个紧跟首选,其余排在颜文字之后
symbol_mixed_input = false
# 云候选
cloud_candidates = true
# TODO: 极简模式。极简模式下,候选窗口使用 d2d 绘制,界面只作展示候选项,无其他功能。
@@ -533,8 +535,8 @@ quick_phrase = true # Shift+K 进入快捷短语模式
quick_phrase_candidates = false # 全拼/双拼下把编码精确匹配的快捷短语混进候选,排在固定位置候选之后的首位
quick_phrase_frequency = true # 快捷短语参与调频:常越过它选别的候选时整组后退,最多退到首页末位
# 候选混输总开关:关闭后英文(general.cn_en_mixed_input)、emoji(general.emoji_mixed_input)、
# 颜文字(general.kaomoji_mixed_input)、快捷短语(quick_phrase_candidates)、日期时间(date_time_candidates)
# 都不再混进候选,各自的开关值保留
# 颜文字(general.kaomoji_mixed_input)、符号(general.symbol_mixed_input)、快捷短语(quick_phrase_candidates)、
# 日期时间(date_time_candidates)都不再混进候选,各自的开关值保留
mixed_candidates = true
date_time_candidates = false # 全拼/双拼下输入 rq / riqi / date、sj / shijian / time、xq / xingqi / week 时,把当前日期、时间、星期混进候选
date_time_menu = true # 日期时间混入候选时,在第三位放一个「📅日期」入口,选中后候选框换成全部日期格式(与 Shift+T 模式相同)
@@ -542,6 +544,7 @@ radical_mode = true # Shift+U 进入 U 模式(部件拆字反查):输入
date_time_mode = true # Shift+T 进入日期与时间快捷输入模式(全拼/双拼/五笔)
emoji_mode = true # Shift+E 进入 Emoji 快捷输入模式(支持全拼/简拼/双拼/英文)
kaomoji_mode = true # Shift+M 进入颜文字快捷输入模式(支持全拼/简拼/双拼/英文)
symbol_mode = true # Shift+S 进入符号快捷输入模式:输入分组码(xl 希腊字母、sx 数学、szq 带圈数字、jt 箭头)或符号名(alpha、infinity)
jianpin_mode = true # Shift+J 进入超级简拼模式(每个字母作为简拼;双拼按方案转换 zh/ch/sh)
y_mode = true # Shift+Y 进入临时英文模式(中文状态下直接打英文,上屏后回到中文)
r_mode = true # Shift+R 进入临时日语模式(中文状态下使用罗马字输入日语,上屏后回到中文)
+4
View File
@@ -162,6 +162,7 @@ double g_assoc_sentence_collocation_rear_penalty = -18.0;
bool g_assoc_sentence_source_badge = false;
bool g_emoji_mixed_input_enabled = false;
bool g_kaomoji_mixed_input_enabled = false;
bool g_symbol_mixed_input_enabled = false;
bool g_radical_mode_enabled = true;
bool g_quick_phrase_enabled = true;
bool g_quick_phrase_candidates_enabled = false;
@@ -172,6 +173,7 @@ bool g_date_time_menu_enabled = true;
bool g_date_time_mode_enabled = true;
bool g_emoji_mode_enabled = false;
bool g_kaomoji_mode_enabled = false;
bool g_symbol_mode_enabled = true;
bool g_jianpin_mode_enabled = true;
bool g_y_mode_enabled = true;
bool g_r_mode_enabled = true;
@@ -620,6 +622,7 @@ bool LoadImeConfig()
g_assoc_sentence_source_badge = tbl["association"]["sentence_source_badge"].value_or(false);
g_emoji_mixed_input_enabled = tbl["general"]["emoji_mixed_input"].value_or(false);
g_kaomoji_mixed_input_enabled = tbl["general"]["kaomoji_mixed_input"].value_or(false);
g_symbol_mixed_input_enabled = tbl["general"]["symbol_mixed_input"].value_or(false);
// 改名前的 unicode_mode 作迁移兜底;升级合并已把它搬到新键,新写入只用新键。
g_radical_mode_enabled = tbl["utility"]["radical_mode"].value_or(tbl["utility"]["unicode_mode"].value_or(true));
g_quick_phrase_enabled = tbl["utility"]["quick_phrase"].value_or(true);
@@ -631,6 +634,7 @@ bool LoadImeConfig()
g_date_time_mode_enabled = tbl["utility"]["date_time_mode"].value_or(true);
g_emoji_mode_enabled = tbl["utility"]["emoji_mode"].value_or(false);
g_kaomoji_mode_enabled = tbl["utility"]["kaomoji_mode"].value_or(false);
g_symbol_mode_enabled = tbl["utility"]["symbol_mode"].value_or(true);
g_jianpin_mode_enabled = tbl["utility"]["jianpin_mode"].value_or(true);
g_y_mode_enabled = tbl["utility"]["y_mode"].value_or(true);
g_r_mode_enabled = tbl["utility"]["r_mode"].value_or(true);
+6
View File
@@ -478,6 +478,9 @@ bool GetConfiguredEmojiMixedInputEnabled();
bool SetConfiguredEmojiMixedInputEnabled(bool enabled);
bool GetConfiguredKaomojiMixedInputEnabled();
bool SetConfiguredKaomojiMixedInputEnabled(bool enabled);
// 全拼/双拼里输入恰好等于某个符号编码(alpha、szq、xl)时,把符号混进候选。
bool GetConfiguredSymbolMixedInputEnabled();
bool SetConfiguredSymbolMixedInputEnabled(bool enabled);
bool GetConfiguredCloudCandidatesEnabled();
bool SetConfiguredCloudCandidatesEnabled(bool enabled);
// 整句候选来源与去重补位开关([association] 段),默认全关。
@@ -533,6 +536,9 @@ bool GetConfiguredEmojiModeEnabled();
bool SetConfiguredEmojiModeEnabled(bool enabled);
bool GetConfiguredKaomojiModeEnabled();
bool SetConfiguredKaomojiModeEnabled(bool enabled);
// Shift+S 符号快捷输入模式。
bool GetConfiguredSymbolModeEnabled();
bool SetConfiguredSymbolModeEnabled(bool enabled);
bool GetConfiguredJianpinModeEnabled();
bool SetConfiguredJianpinModeEnabled(bool enabled);
bool GetConfiguredYModeEnabled();
+30
View File
@@ -165,6 +165,21 @@ bool SetConfiguredKaomojiMixedInputEnabled(bool enabled)
return true;
}
bool GetConfiguredSymbolMixedInputEnabled()
{
return g_symbol_mixed_input_enabled;
}
bool SetConfiguredSymbolMixedInputEnabled(bool enabled)
{
if (!WriteConfiguredValue("general", "symbol_mixed_input", enabled ? "true" : "false"))
{
return false;
}
g_symbol_mixed_input_enabled = enabled;
return true;
}
bool GetConfiguredCloudCandidatesEnabled()
{
return g_cloud_candidates_enabled;
@@ -486,6 +501,21 @@ bool SetConfiguredKaomojiModeEnabled(bool enabled)
return true;
}
bool GetConfiguredSymbolModeEnabled()
{
return g_symbol_mode_enabled;
}
bool SetConfiguredSymbolModeEnabled(bool enabled)
{
if (!WriteConfiguredValue("utility", "symbol_mode", enabled ? "true" : "false"))
{
return false;
}
g_symbol_mode_enabled = enabled;
return true;
}
bool GetConfiguredJianpinModeEnabled()
{
return g_jianpin_mode_enabled;
+2
View File
@@ -205,6 +205,7 @@ extern double g_assoc_sentence_collocation_weak_penalty;
extern double g_assoc_sentence_collocation_rear_penalty;
extern bool g_emoji_mixed_input_enabled;
extern bool g_kaomoji_mixed_input_enabled;
extern bool g_symbol_mixed_input_enabled;
extern bool g_radical_mode_enabled;
extern bool g_quick_phrase_enabled;
extern bool g_quick_phrase_candidates_enabled;
@@ -215,6 +216,7 @@ extern bool g_date_time_menu_enabled;
extern bool g_date_time_mode_enabled;
extern bool g_emoji_mode_enabled;
extern bool g_kaomoji_mode_enabled;
extern bool g_symbol_mode_enabled;
extern bool g_jianpin_mode_enabled;
extern bool g_y_mode_enabled;
extern bool g_r_mode_enabled;
+63 -3
View File
@@ -17,6 +17,7 @@
#include <filesystem>
#include <limits>
#include <string>
#include <unordered_set>
namespace msimeui
{
@@ -233,6 +234,26 @@ std::wstring Lower(std::wstring value)
return value;
}
// 关键词以空格分隔。整词等于搜索词才算:搜 pi 时 π 的关键词里有 pi,而「偏旁 pianpang」只是含有 pi。
bool HasKeyword(const std::wstring &keywordsLower, const std::wstring &query)
{
size_t start = 0;
while (true)
{
const size_t end = keywordsLower.find(L' ', start);
const size_t length = (end == std::wstring::npos ? keywordsLower.size() : end) - start;
if (length == query.size() && keywordsLower.compare(start, length, query) == 0)
{
return true;
}
if (end == std::wstring::npos)
{
return false;
}
start = end + 1;
}
}
bool CopyToClipboard(HWND hwnd, const std::wstring &text)
{
if (!OpenClipboard(hwnd))
@@ -1222,6 +1243,29 @@ std::vector<const EmojiPanel::Item *> EmojiPanel::CollectDiversePreviewItems(con
return items;
}
std::vector<const EmojiPanel::Item *> EmojiPanel::CollectBestSymbolMatches(const std::wstring &query,
size_t limit) const
{
// 同一个符号可能在几个小标题下各有一份(拉丁扩展与拼音都收了 ā),最佳匹配里只列一次。
std::vector<const Item *> items;
std::unordered_set<std::wstring> seen;
for (const auto &group : symbolGroups_)
{
for (const auto &item : group.items)
{
if (HasKeyword(item.keywordsLower, query) && seen.insert(item.text).second)
{
items.push_back(&item);
if (items.size() >= limit)
{
return items;
}
}
}
}
return items;
}
size_t EmojiPanel::HitMainTab(const PointF &point) const
{
if (InDetailPage())
@@ -1482,13 +1526,20 @@ void EmojiPanel::EnsureDisplayLayout() const
auto symbolPreview = CollectDiversePreviewItems(symbolGroups_, kPreviewLimit);
if (searching)
{
symbolPreview.clear();
// 整词命中的排前面,不够一页再补子串命中的。
symbolPreview = CollectBestSymbolMatches(query, kPreviewLimit);
for (const auto &group : symbolGroups_)
{
if (symbolPreview.size() >= kPreviewLimit)
{
break;
}
for (const auto &item : group.items)
{
if (item.keywordsLower.find(query) != std::wstring::npos ||
item.text.find(searchText_) != std::wstring::npos)
if ((item.keywordsLower.find(query) != std::wstring::npos ||
item.text.find(searchText_) != std::wstring::npos) &&
std::none_of(symbolPreview.begin(), symbolPreview.end(),
[&item](const Item *shown) { return shown->text == item.text; }))
{
symbolPreview.push_back(&item);
if (symbolPreview.size() >= kPreviewLimit)
@@ -1535,6 +1586,15 @@ void EmojiPanel::EnsureDisplayLayout() const
{
if (searching || symbolTabs_.empty())
{
// 关键词子串命中面很宽(搜 pi 会带上「偏旁 pianpang」整组),整词命中的单独列在最前。
if (searching)
{
auto best = CollectBestSymbolMatches(query, kColumns * 10);
if (!best.empty())
{
appendGroup(L"最佳匹配", std::move(best), Page::Home, false);
}
}
for (const auto &group : symbolGroups_)
{
auto items = filterItems(group.items);
+2
View File
@@ -151,6 +151,8 @@ class EmojiPanel final : public Visual
const SymbolTab *ActiveSymbolTab() const;
std::vector<const Item *> CollectPreviewItems(const std::vector<Group> &groups, size_t limit) const;
std::vector<const Item *> CollectDiversePreviewItems(const std::vector<Group> &groups, size_t limit) const;
// 关键词里有一整个词等于搜索词的符号(alpha → α、pi → π、xl → 希腊字母),按目录顺序、不重复。
std::vector<const Item *> CollectBestSymbolMatches(const std::wstring &query, size_t limit) const;
std::vector<Group> emojiGroups_;
std::vector<Group> kaomojiGroups_;
@@ -22,6 +22,7 @@ inline bool IsUiOnlyCandidateSource(CandidateSource source)
case CandidateSource::EnglishDictionary:
case CandidateSource::Emoji:
case CandidateSource::Kaomoji:
case CandidateSource::Symbol:
case CandidateSource::DateTime:
return true;
default:
+18 -4
View File
@@ -141,14 +141,15 @@ inline std::optional<size_t> SlottedEnglishIndex(const std::vector<WordItem> &it
// (promoted ahead of AI unless a cloud result forces it behind cloud+AI), and
// emoji/kaomoji are placed at the end of the candidate list so they do not
// displace normal Chinese candidates:
// no cloud: Chinese..., English, AI, ..., emoji, kaomoji
// cloud: Chinese..., cloud, AI, English, ..., emoji, kaomoji
// cloud only: Chinese..., cloud, English, ..., emoji, kaomoji
// base: Chinese..., English, ..., emoji, kaomoji
// no cloud: Chinese..., English, AI, ..., emoji, kaomoji, symbols
// cloud: Chinese..., cloud, AI, English, ..., emoji, kaomoji, symbols
// cloud only: Chinese..., cloud, English, ..., emoji, kaomoji, symbols
// base: Chinese..., English, ..., emoji, kaomoji, symbols
// 知道每页候选数(page_size)时,最匹配的那个 emoji 单独放在首页末位,其余 emoji 仍在末尾;
// 补全词的默认位置和学到的英文槽位相应让到首页倒数第二位,两者都留在首页。
// 日期时间只在 rq / sj / xq 这类唤醒词上出现,用户打它就是要日期,所以它排在所有异步候选前面,
// 紧跟首个中文候选,展开全部格式的入口紧跟在它后面:Chinese, date/time, 📅日期, cloud, ...
// 符号同理(只在整串等于编码时出现),首个符号排在日期时间之后、云候选之前,其余在末尾。
// The learned English slot (EnglishPlacement) moves the English candidate away
// from that default afterwards, and explicit English ranking choices are
// reapplied last.
@@ -160,6 +161,7 @@ inline void NormalizeMixedCandidateOrder(std::vector<WordItem> &items, size_t lo
std::vector<WordItem> fixed_english_candidates;
std::vector<WordItem> emoji_candidates;
std::vector<WordItem> kaomoji_candidates;
std::vector<WordItem> symbol_candidates;
std::vector<WordItem> date_time_candidates;
std::optional<WordItem> date_time_menu;
std::optional<WordItem> cloud_candidate;
@@ -193,6 +195,9 @@ inline void NormalizeMixedCandidateOrder(std::vector<WordItem> &items, size_t lo
case CandidateSource::Kaomoji:
kaomoji_candidates.push_back(std::move(item));
break;
case CandidateSource::Symbol:
symbol_candidates.push_back(std::move(item));
break;
case CandidateSource::DateTime:
if (!metasequoia::local_modes::is_date_time_menu_item(item))
date_time_candidates.push_back(std::move(item));
@@ -234,6 +239,13 @@ inline void NormalizeMixedCandidateOrder(std::vector<WordItem> &items, size_t lo
if (date_time_menu)
insert_at(slot++, std::move(*date_time_menu));
}
// 符号只在整串等于某个编码时混进来(alpha、szq、yi),用户打的就是它:首个紧跟首个中文候选
// (和日期时间一样),其余排在颜文字之后。
if (!symbol_candidates.empty())
{
insert_at(slot++, std::move(symbol_candidates.front()));
symbol_candidates.erase(symbol_candidates.begin());
}
if (cloud_candidate)
{
insert_at(slot++, std::move(*cloud_candidate));
@@ -305,6 +317,8 @@ inline void NormalizeMixedCandidateOrder(std::vector<WordItem> &items, size_t lo
items.push_back(std::move(candidate));
for (auto &candidate : kaomoji_candidates)
items.push_back(std::move(candidate));
for (auto &candidate : symbol_candidates)
items.push_back(std::move(candidate));
for (auto &candidate : date_time_candidates)
items.push_back(std::move(candidate));
+13 -3
View File
@@ -48,6 +48,7 @@ bool g_radical_mode_triggered = false;
bool g_date_time_mode_triggered = false;
bool g_emoji_mode_triggered = false;
bool g_kaomoji_mode_triggered = false;
bool g_symbol_mode_triggered = false;
bool g_jianpin_mode_triggered = false;
bool g_y_mode_triggered = false;
bool g_r_mode_triggered = false;
@@ -274,6 +275,14 @@ bool IsKaomojiCompositionActive(const std::string &raw)
});
}
// S 模式:符号编码只有字母(万象分组码与 codex 符号名),大写按小写查。
bool IsSymbolCompositionActive(const std::string &raw)
{
return g_symbol_mode_triggered && !raw.empty() && raw.front() == 'S' &&
std::all_of(raw.begin() + 1, raw.end(),
[](unsigned char ch) { return (ch >= 'a' && ch <= 'z') || (ch >= 'A' && ch <= 'Z'); });
}
bool IsJianpinCompositionActive(const std::string &raw)
{
return g_jianpin_mode_triggered && !raw.empty() && raw.front() == 'J';
@@ -310,19 +319,20 @@ void ClearSpecialModeTriggers()
g_date_time_mode_triggered = false;
g_emoji_mode_triggered = false;
g_kaomoji_mode_triggered = false;
g_symbol_mode_triggered = false;
g_jianpin_mode_triggered = false;
g_y_mode_triggered = false;
g_r_mode_triggered = false;
}
// True whenever a K/U/T/E/M/J/Y special-mode composition is in progress, even when the
// True whenever a K/U/T/E/M/S/J/Y special-mode composition is in progress, even when the
// typed text is not yet a complete keyword/letter sequence. Such input must never
// be interpreted as normal pinyin.
bool IsSpecialModeCompositionActive(const std::string &raw)
{
return IsQuickPhraseCompositionActive(raw) || IsRadicalCompositionActive(raw) || IsDateTimeCompositionActive(raw) ||
IsEmojiCompositionActive(raw) || IsKaomojiCompositionActive(raw) || IsJianpinCompositionActive(raw) ||
IsYModeCompositionActive(raw) || IsVModeCompositionActive(raw);
IsEmojiCompositionActive(raw) || IsKaomojiCompositionActive(raw) || IsSymbolCompositionActive(raw) ||
IsJianpinCompositionActive(raw) || IsYModeCompositionActive(raw) || IsVModeCompositionActive(raw);
}
bool IsWubiLiteralComposition(const std::string &raw)
@@ -400,7 +400,7 @@ void InsertFirstAndAppendRest(std::vector<WordItem> &items, std::vector<WordItem
}
} // namespace
// 混输线程(MixedCandidates)查完一次输入后,把英文、emoji、颜文字、日期时间一并合进当前候选,
// 混输线程(MixedCandidates)查完一次输入后,把英文、emoji、颜文字、符号、日期时间一并合进当前候选,
// 只刷新一次候选窗。各来源的开关在这里再核一遍:请求发出后用户可能刚在设置页关掉了它。
void ApplyMixedCandidates(MixedCandidates::Result result, const std::string &input, uint64_t generation)
{
@@ -464,6 +464,13 @@ void ApplyMixedCandidates(MixedCandidates::Result result, const std::string &inp
AppendCandidates(items, std::move(kaomoji));
}
if (GetConfiguredSymbolMixedInputEnabled())
{
auto symbol = ReplaceSourceAndDeduplicate(items, CandidateSource::Symbol, std::move(result.symbol));
changed = changed || !symbol.empty();
AppendCandidates(items, std::move(symbol));
}
// 全被去重掉、列表也没少东西时,屏幕上的那页就是对的,不必重画。
if (!changed && items.size() == size_before)
return;
+41 -6
View File
@@ -28,6 +28,7 @@
#include "engine/local_modes/date_time_query.h"
#include "engine/local_modes/emoji_query.h"
#include "engine/local_modes/kaomoji_query.h"
#include "engine/local_modes/symbol_query.h"
#include "engine/local_modes/jianpin_query.h"
#include "engine/local_modes/v_mode_query.h"
#include "engine/shuangpin/shuangpin_profile.h"
@@ -60,7 +61,7 @@ bool BuildTranslationQuery(const WordItem &item, EnglishIme::TranslationQuery &q
if (visible.empty())
return false;
if (item.source == CandidateSource::Emoji || item.source == CandidateSource::Kaomoji ||
item.source == CandidateSource::DateTime)
item.source == CandidateSource::Symbol || item.source == CandidateSource::DateTime)
return false;
if (item.source == CandidateSource::EnglishDictionary && !item.pinyin.empty())
{
@@ -160,6 +161,26 @@ std::string DateTimeMixedKeyword(const std::string &current_input, SchemeType sc
return metasequoia::local_modes::is_date_time_keyword(quanpin) ? quanpin : std::string{};
}
// 符号混输按整串编码查。双拼的按键串是音节码,用户要的是它读出来的全拼(小鹤 sj 是 san → ③),
// 所以先查换算出的全拼;全拼查不到再查原始按键,alpha 这类不是音节的编码在双拼下照样能混。
std::vector<std::string> SymbolMixedCodes(const std::string &current_input, SchemeType scheme)
{
if (current_input.empty() || !std::all_of(current_input.begin(), current_input.end(),
[](unsigned char ch) { return ch >= 'a' && ch <= 'z'; }))
return {};
std::vector<std::string> codes;
if (scheme == SchemeType::Shuangpin)
{
std::string quanpin = g_inputSession->get_quanpin();
quanpin.erase(std::remove_if(quanpin.begin(), quanpin.end(), [](char ch) { return ch == '\'' || ch == ' '; }),
quanpin.end());
if (!quanpin.empty() && quanpin != current_input)
codes.push_back(std::move(quanpin));
}
codes.push_back(current_input);
return codes;
}
// U 模式(部件拆字反查):唤醒键 U 之后已经输入了至少一个部件字母才算一次可查询的输入。
bool IsRadicalInput(const std::string &raw)
{
@@ -183,6 +204,11 @@ bool IsKaomojiInput(const std::string &raw)
return IsKaomojiCompositionActive(raw) && raw.size() > 1;
}
bool IsSymbolInput(const std::string &raw)
{
return IsSymbolCompositionActive(raw) && raw.size() > 1;
}
bool IsJianpinInput(const std::string &raw)
{
return IsJianpinCompositionActive(raw) && raw.size() > 1 &&
@@ -306,8 +332,8 @@ std::string BuildCurrentCandidatePage()
}
if (show_helpcodes && item.source != CandidateSource::EnglishDictionary &&
item.source != CandidateSource::QuickPhrase && item.source != CandidateSource::Emoji &&
item.source != CandidateSource::Kaomoji && item.source != CandidateSource::DateTime &&
item.source != CandidateSource::Generated)
item.source != CandidateSource::Kaomoji && item.source != CandidateSource::Symbol &&
item.source != CandidateSource::DateTime && item.source != CandidateSource::Generated)
view.annotation = g_inputSession->get_helpcode_annotation(item.word, uppercase_all_helpcodes);
if (!IsCandidateSubPageActive() && item.word == JevSelectionWord() && Jev::IsPoolSource(item) &&
item.fixed_position == 0)
@@ -704,6 +730,11 @@ void PrepareCandidateList(uint64_t client_id, uint64_t activation_epoch)
10, ConfiguredShuangpinProfile())
.candidates;
}
else if (IsSymbolInput(current_input))
{
// 一个分组可以有上百个符号(jt 箭头 148 个),整组都放进列表,靠翻页看全。
items = metasequoia::local_modes::query_symbols(current_input.substr(1)).candidates;
}
else if (IsJianpinInput(current_input))
{
const int limit = current_input.size() == 2 ? 24 : 100;
@@ -730,8 +761,8 @@ void PrepareCandidateList(uint64_t client_id, uint64_t activation_epoch)
}
else if (IsSpecialModeCompositionActive(current_input))
{
// A K/U/T/E/M/J/Y special-mode prefix that is not yet a complete input (e.g.
// "K", "U", "U+", "Tw", "Txin", "E", "M", "J", "Y"): do not translate it into
// A K/U/T/E/M/S/J/Y special-mode prefix that is not yet a complete input (e.g.
// "K", "U", "U+", "Tw", "Txin", "E", "M", "S", "J", "Y"): do not translate it into
// normal pinyin candidates. Leave items empty so only the raw typed text
// shows as the fallback.
}
@@ -798,9 +829,13 @@ void PrepareCandidateList(uint64_t client_id, uint64_t activation_epoch)
mixed.english_first_page_count = static_cast<size_t>(GetConfiguredEnglishMixedInputFirstPageCount());
mixed.emoji = GetConfiguredEmojiMixedInputEnabled();
mixed.kaomoji = GetConfiguredKaomojiMixedInputEnabled();
// 符号按整串编码精确匹配,光标前缀截断时整串不是用户要的编码。
if (GetConfiguredSymbolMixedInputEnabled() && g_inputSession->prefix_end() == current_input.size())
mixed.symbol_codes = SymbolMixedCodes(current_input, scheme);
if (GetConfiguredDateTimeCandidatesEnabled() && g_inputSession->prefix_end() == current_input.size())
mixed.date_time_keyword = DateTimeMixedKeyword(current_input, scheme);
if (mixed.english || mixed.emoji || mixed.kaomoji || !mixed.date_time_keyword.empty())
if (mixed.english || mixed.emoji || mixed.kaomoji || !mixed.symbol_codes.empty() ||
!mixed.date_time_keyword.empty())
{
mixed.input = current_input;
mixed.scheme = scheme;
+3
View File
@@ -40,6 +40,7 @@ extern bool g_radical_mode_triggered;
extern bool g_date_time_mode_triggered;
extern bool g_emoji_mode_triggered;
extern bool g_kaomoji_mode_triggered;
extern bool g_symbol_mode_triggered;
extern bool g_jianpin_mode_triggered;
extern bool g_y_mode_triggered;
extern bool g_r_mode_triggered;
@@ -88,6 +89,8 @@ bool IsRadicalCompositionActive(const std::string &raw);
bool IsDateTimeCompositionActive(const std::string &raw);
bool IsEmojiCompositionActive(const std::string &raw);
bool IsKaomojiCompositionActive(const std::string &raw);
// S 模式(符号快捷输入):唤醒键 S 之后是一串字母编码。
bool IsSymbolCompositionActive(const std::string &raw);
bool IsJianpinCompositionActive(const std::string &raw);
bool IsYModeCompositionActive(const std::string &raw);
bool IsYModeInput(const std::string &raw);
+11 -3
View File
@@ -34,8 +34,8 @@ namespace
bool IsShiftLetterSpecialModeTriggered()
{
return g_quick_phrase_triggered || g_radical_mode_triggered || g_date_time_mode_triggered ||
g_emoji_mode_triggered || g_kaomoji_mode_triggered || g_jianpin_mode_triggered || g_y_mode_triggered ||
g_r_mode_triggered;
g_emoji_mode_triggered || g_kaomoji_mode_triggered || g_symbol_mode_triggered || g_jianpin_mode_triggered ||
g_y_mode_triggered || g_r_mode_triggered;
}
// 字母键写进 raw 的那个字母:键产生了字母就照收(保留大小写),否则取键位本身的小写字母。
@@ -1056,7 +1056,7 @@ void HandleImeKey(uint64_t client_id, uint64_t activation_epoch, uint64_t reques
const std::optional<SchemeType> scheme =
g_inputSession ? std::optional<SchemeType>(g_inputSession->current_scheme_type()) : std::nullopt;
const bool chinese_scheme = scheme == SchemeType::Quanpin || scheme == SchemeType::Shuangpin;
// T 模式对五笔同样开放,但五笔不能并进 chinese_scheme:那个变量还给 K/U/E/M/J/Y/R 触发与
// T 模式对五笔同样开放,但五笔不能并进 chinese_scheme:那个变量还给 K/U/E/M/S/J/Y/R 触发与
// ShouldLearnEnteredEnglishWord 用,并进去会让五笔连带打开其它本地模式(R7)。
const bool date_time_scheme = chinese_scheme || scheme == SchemeType::Wubi;
const bool pinyin_commit =
@@ -1101,6 +1101,9 @@ void HandleImeKey(uint64_t client_id, uint64_t activation_epoch, uint64_t reques
if (chinese_scheme && !g_english_input_mode && GetConfiguredKaomojiModeEnabled() && input_before_key.empty() &&
Global::Keycode == 'M' && Global::Wch == L'M' && shift_only)
g_kaomoji_mode_triggered = true;
if (chinese_scheme && !g_english_input_mode && GetConfiguredSymbolModeEnabled() && input_before_key.empty() &&
Global::Keycode == 'S' && Global::Wch == L'S' && shift_only)
g_symbol_mode_triggered = true;
if (chinese_scheme && !g_english_input_mode && GetConfiguredJianpinModeEnabled() && input_before_key.empty() &&
Global::Keycode == 'J' && Global::Wch == L'J' && shift_only)
g_jianpin_mode_triggered = true;
@@ -1318,6 +1321,11 @@ void HandleImeKey(uint64_t client_id, uint64_t activation_epoch, uint64_t reques
// Keep preedit identical to the typed M-prefixed code.
GlobalIme::composition.segmented_pinyin = GlobalIme::composition.raw_input_with_cases;
}
if (!g_english_input_mode && IsSymbolCompositionActive(GlobalIme::composition.raw_input_with_cases))
{
// Keep preedit identical to the typed S-prefixed code.
GlobalIme::composition.segmented_pinyin = GlobalIme::composition.raw_input_with_cases;
}
if (!g_english_input_mode && IsJianpinCompositionActive(GlobalIme::composition.raw_input_with_cases))
{
// Keep preedit identical to the typed J-prefixed code.
+5 -3
View File
@@ -91,9 +91,10 @@ void LearnEnglishSlot(const WordItem &selected, size_t absolute_index, uint64_t
activation_epoch);
return;
}
// emoji、颜文字按自己的位置排(最匹配的 emoji 就在首页末位),选它们不是嫌英文排得太前,不让英文后退。
// emoji、颜文字、符号按自己的位置排(最匹配的 emoji 就在首页末位),选它们不是嫌英文排得太前,不让英文后退。
if (absolute_index > *english_index && selected.source != CandidateSource::EnglishDictionary &&
selected.source != CandidateSource::Emoji && selected.source != CandidateSource::Kaomoji)
selected.source != CandidateSource::Emoji && selected.source != CandidateSource::Kaomoji &&
selected.source != CandidateSource::Symbol)
{
FanyNamedPipe::EnqueueLearnEnglishSlotTask(code, static_cast<int>(*english_index),
static_cast<int>(absolute_index), client_id, activation_epoch);
@@ -289,7 +290,8 @@ void ProcessSelectionKey(UINT keycode, uint64_t client_id, uint64_t activation_e
// 日语假名可能只覆盖已完成的音节,和词典候选一起推进,保留尚未转换的后缀。
if (curWordItem.source == CandidateSource::EnglishDictionary ||
curWordItem.source == CandidateSource::QuickPhrase || curWordItem.source == CandidateSource::Emoji ||
curWordItem.source == CandidateSource::Kaomoji || curWordItem.source == CandidateSource::DateTime ||
curWordItem.source == CandidateSource::Kaomoji || curWordItem.source == CandidateSource::Symbol ||
curWordItem.source == CandidateSource::DateTime ||
(curWordItem.source == CandidateSource::Generated && !japanese_candidate))
{
Global::candidate_ui.selected_text =
+2 -1
View File
@@ -651,7 +651,8 @@ void WorkerThread()
break;
}
if (item.source == CandidateSource::QuickPhrase || item.source == CandidateSource::Emoji ||
item.source == CandidateSource::Kaomoji || item.source == CandidateSource::Generated)
item.source == CandidateSource::Kaomoji || item.source == CandidateSource::Symbol ||
item.source == CandidateSource::Generated)
{
break;
}
+15 -1
View File
@@ -6,6 +6,7 @@
#include "engine/local_modes/date_time_query.h"
#include "engine/local_modes/emoji_query.h"
#include "engine/local_modes/kaomoji_query.h"
#include "engine/local_modes/symbol_query.h"
#include "engine/shuangpin/shuangpin_profile.h"
#include "config/ime_config.h"
#include <algorithm>
@@ -18,7 +19,7 @@
namespace
{
constexpr std::size_t kEnglishCandidateLimit = 5;
// emoji 和颜文字从第二个字母起才查,单独一个首字母匹配面太大,会把候选刷满。
// emoji、颜文字和符号从第二个字母起才查,单独一个首字母匹配面太大,会把候选刷满。
constexpr std::size_t kExpressiveMinimumPrefixLength = 2;
constexpr int kExpressiveCandidateLimit = 3;
constexpr std::size_t kDateTimeCandidateLimit = 3;
@@ -118,6 +119,19 @@ void WorkerLoop()
request.input, request.scheme, others_db_path, kExpressiveCandidateLimit,
GetShuangpinProfile(GetConfiguredShuangpinSchema()))
.candidates;
if (IsStale(observed_generation))
continue;
}
if (expressive_prefix)
{
for (const std::string &code : request.symbol_codes)
{
result.symbol =
metasequoia::local_modes::query_mixed_symbols(code, others_db_path, kExpressiveCandidateLimit)
.candidates;
if (!result.symbol.empty())
break;
}
}
// 什么都没查到就不回调:任务线程组页时已经是不带混输候选的列表,没有可以合并的。
+6 -2
View File
@@ -8,7 +8,7 @@
#include <string>
#include <vector>
// 候选混输的后台查询:英文、emoji、颜文字、日期时间共用这一条线程。它们各自都只是一次小查询,
// 候选混输的后台查询:英文、emoji、颜文字、符号、日期时间共用这一条线程。它们各自都只是一次小查询,
// 合在一起按一次输入查完、一次回调交回任务线程,候选窗每个按键最多因混输刷新一次。
// 专用英文模式(英文状态、Shift+Y)和候选译文仍走 EnglishIme,快捷短语在任务线程上同步混入。
namespace MixedCandidates
@@ -24,6 +24,9 @@ struct Request
std::size_t english_first_page_count = 1;
bool emoji = false;
bool kaomoji = false;
// 符号只认整串精确等于编码、且建库时标了可混输的行(alpha、szq、yi → ①),和 emoji 的前缀匹配不同。
// 按顺序试,取第一个查得到的:双拼先放换算出的全拼(sj → san),再放原始按键;空表示这次不查。
std::vector<std::string> symbol_codes;
// 日期时间唤醒词(rq / riqi / sj ...),空串表示这次不查。双拼要先换算成全拼才认得出
// shijian 这类整拼,换算依赖会话,所以在任务线程上算好再交过来。
std::string date_time_keyword;
@@ -34,11 +37,12 @@ struct Result
std::vector<WordItem> english;
std::vector<WordItem> emoji;
std::vector<WordItem> kaomoji;
std::vector<WordItem> symbol;
std::vector<WordItem> date_time;
bool empty() const
{
return english.empty() && emoji.empty() && kaomoji.empty() && date_time.empty();
return english.empty() && emoji.empty() && kaomoji.empty() && symbol.empty() && date_time.empty();
}
};
@@ -141,6 +141,7 @@ VisualPtr BuildToolsPage(PageContext &context)
auto zhEn = BindToggle(context, "general.cn_en_mixed_input");
auto emoji = BindToggle(context, "general.emoji_mixed_input");
auto kaomoji = BindToggle(context, "general.kaomoji_mixed_input");
auto symbol = BindToggle(context, "general.symbol_mixed_input");
auto quickCandidates = BindToggle(context, "utility.quick_phrase_candidates");
auto quickFrequency = BindToggle(context, "utility.quick_phrase_frequency");
auto dateTime = BindToggle(context, "utility.date_time_candidates");
@@ -149,7 +150,7 @@ VisualPtr BuildToolsPage(PageContext &context)
frequency.minWidth = 110.0f;
auto trigger = BindDropdown(context, "general.cn_en_mixed_input_min_chars", NumberOptions(1, 8), frequency,
ValueKind::Integer);
std::vector<std::shared_ptr<ToggleSwitch>> subs = {zhEn, emoji, kaomoji, quickCandidates,
std::vector<std::shared_ptr<ToggleSwitch>> subs = {zhEn, emoji, kaomoji, symbol, quickCandidates,
quickFrequency, dateTime, dateMenu};
std::shared_ptr<Hideable> hint;
ToggleSection section = MakeToggleSection(
@@ -180,6 +181,10 @@ VisualPtr BuildToolsPage(PageContext &context)
L"在候选项中加入匹配的 emoji(排在候选项列表最后,不挤占正常汉字候选的位置)", emoji));
details.Add(
DetailsRow(context, L"颜文字混输", L"在候选项中加入匹配的颜文字(排在列表末尾的 emoji 之后)", kaomoji));
details.Add(DetailsRow(context, L"符号混输",
L"输入恰好等于某个符号编码时加入符号,首个紧跟首选,例如 alpha → α、infinity → "
L"∞、szq → ①、yi → ① ⑴ ⒈。其他读得成拼音的编码(xi、ji)只在 S 模式里用",
symbol));
details.Add(
DetailsRow(context, L"快捷短语混入候选",
L"输入的编码与快捷短语的编码完全一致时,快捷短语直接出现在候选首位(固定位置的候选之后),"
@@ -273,6 +278,12 @@ VisualPtr BuildToolsPage(PageContext &context)
L"中文模式下按 Shift+M,再输入全拼 / 简拼 / 双拼 / 英文关键词即可输入颜"
L"文字。例如 haixiu、hx、zai xiang、zx、kiss。空格上屏;数字选词。"),
BindToggle(context, "utility.kaomoji_mode"), 2.0f));
page.AddSection(std::make_shared<TopAlignedRow>(
QuickTitleBlock(L"符号快捷输入(S 模式)",
L"中文模式下按 Shift+S,再输入分组码或符号名即可输入符号。例如 xl 希腊字母、sx "
L"数学符号、szq 带圈数字、lmd 罗马数字、jt 箭头、yi 数字 1 的各种写法、alpha、infinity。"
L"空格上屏;数字选词。"),
BindToggle(context, "utility.symbol_mode"), 2.0f));
const auto centered = [&context](const std::wstring &title, const std::wstring &description,
const std::string &path) {
+6
View File
@@ -518,6 +518,7 @@ std::wstring BuildConfigMessage(bool refresh_skin_catalog)
{"cn_en_mixed_input_first_page_count", GetConfiguredEnglishMixedInputFirstPageCount()},
{"emoji_mixed_input", GetConfiguredEmojiMixedInputEnabled()},
{"kaomoji_mixed_input", GetConfiguredKaomojiMixedInputEnabled()},
{"symbol_mixed_input", GetConfiguredSymbolMixedInputEnabled()},
{"cloud_candidates", GetConfiguredCloudCandidatesEnabled()},
{"paging_minus_equal", GetConfiguredPagingMinusEqualEnabled()},
{"paging_comma_period", GetConfiguredPagingCommaPeriodEnabled()},
@@ -583,6 +584,7 @@ std::wstring BuildConfigMessage(bool refresh_skin_catalog)
{"date_time_mode", GetConfiguredDateTimeModeEnabled()},
{"emoji_mode", GetConfiguredEmojiModeEnabled()},
{"kaomoji_mode", GetConfiguredKaomojiModeEnabled()},
{"symbol_mode", GetConfiguredSymbolModeEnabled()},
{"jianpin_mode", GetConfiguredJianpinModeEnabled()},
{"y_mode", GetConfiguredYModeEnabled()},
{"r_mode", GetConfiguredRModeEnabled()},
@@ -933,6 +935,8 @@ bool ApplyConfigUpdate(const json::object &data)
return SetConfiguredEmojiMixedInputEnabled(json::value_to<bool>(data.at("value")));
if (path == "general.kaomoji_mixed_input")
return SetConfiguredKaomojiMixedInputEnabled(json::value_to<bool>(data.at("value")));
if (path == "general.symbol_mixed_input")
return SetConfiguredSymbolMixedInputEnabled(json::value_to<bool>(data.at("value")));
if (path == "general.cloud_candidates")
return SetConfiguredCloudCandidatesEnabled(json::value_to<bool>(data.at("value")));
if (path == "association.sentence_wordlattice")
@@ -971,6 +975,8 @@ bool ApplyConfigUpdate(const json::object &data)
return SetConfiguredEmojiModeEnabled(json::value_to<bool>(data.at("value")));
if (path == "utility.kaomoji_mode")
return SetConfiguredKaomojiModeEnabled(json::value_to<bool>(data.at("value")));
if (path == "utility.symbol_mode")
return SetConfiguredSymbolModeEnabled(json::value_to<bool>(data.at("value")));
if (path == "utility.jianpin_mode")
return SetConfiguredJianpinModeEnabled(json::value_to<bool>(data.at("value")));
if (path == "utility.y_mode")
@@ -53,6 +53,7 @@ inline constexpr bool CountsTowardFirstWordSelection(const WordItem &item)
case CandidateSource::EnglishDictionary:
case CandidateSource::Emoji:
case CandidateSource::Kaomoji:
case CandidateSource::Symbol:
case CandidateSource::DateTime:
return false;
}
@@ -587,6 +587,42 @@ TEST_CASE(KaomojiMixedCandidatesFollowEmojiAtTheEnd)
REQUIRE_EQ(items[5].source, CandidateSource::Kaomoji);
}
TEST_CASE(FirstSymbolMixedCandidateFollowsTheFirstCandidate)
{
const auto local = [](std::string word) { return WordItem("yi", std::move(word), 100); };
const auto emoji = [] { return WordItem("yi", "\xF0\x9F\x98\x80", 1, CandidateSource::Emoji); };
const auto kaomoji = [] { return WordItem("yi", "(^_^)", 1, CandidateSource::Kaomoji); };
const auto symbol = [](std::string word) { return WordItem("yi", std::move(word), 1, CandidateSource::Symbol); };
const auto cloud = [] { return WordItem("yi", "云候选", 1, CandidateSource::CloudSuggestion); };
FanyImeIpc::EnglishPlacement placement;
placement.page_size = 5;
// 首个符号紧跟首个中文候选,其余符号排在最后。
std::vector<WordItem> items = {local("一"), symbol("①"), symbol("⑴"), local("以"),
local("已"), local("意"), local("义"), local("亿")};
FanyImeIpc::NormalizeMixedCandidateOrder(items, 1, placement);
REQUIRE_EQ(items.size(), size_t{8});
REQUIRE_EQ(items[0].word, std::string("一"));
REQUIRE_EQ(items[1].word, std::string("①"));
REQUIRE_EQ(items[2].word, std::string("以"));
REQUIRE_EQ(items.back().word, std::string("⑴"));
// 有 emoji 和云候选时:符号仍在第二位、云候选之前;首页末位归 emoji;其余符号排在颜文字之后。
items = {local("一"), symbol("①"), symbol("⑴"), kaomoji(), emoji(), cloud(), local("以"), local("已"), local("意")};
FanyImeIpc::NormalizeMixedCandidateOrder(items, 1, placement);
REQUIRE_EQ(items[1].word, std::string("①"));
REQUIRE_EQ(items[2].source, CandidateSource::CloudSuggestion);
REQUIRE_EQ(items[4].source, CandidateSource::Emoji);
REQUIRE_EQ(items[items.size() - 2].source, CandidateSource::Kaomoji);
REQUIRE_EQ(items.back().word, std::string("⑴"));
// 再归一化一次位置不变(异步结果回来、翻页扩展都会重排)。
const auto before = items;
FanyImeIpc::NormalizeMixedCandidateOrder(items, 1, placement);
for (size_t index = 0; index < items.size(); ++index)
REQUIRE_EQ(items[index].word, before[index].word);
}
TEST_CASE(DateTimeMixedCandidateLeadsAsyncCandidates)
{
const auto local = [](std::string word) { return WordItem("rq", std::move(word), 100); };