feat: Telegram Bridge 出站消息 Markdown 渲染
新增三层 Markdown → Telegram HTML 渲染管线： markdown/ir.ts — markdown-it 解析 Markdown 为中间表示 (MarkdownIR) - 支持 bold/italic/strikethrough/code/code_block/blockquote/links/lists/headings/tables/hr - 表格使用 code-block 模式渲染为 ASCII 表格（<pre> 保留对齐） - HTML 内联 <br> 标签转换为换行符 markdown/render.ts — 通用标记渲染器 - boundary tracking + LIFO stack 处理嵌套样式 markdown/telegram.ts — Telegram 专用渲染器 - 样式映射到 Telegram HTML 标签 (b/i/s/code/pre+code/blockquote/a) - 文件引用保护：README.md/main.go 等不被误识别为 URL - render-first 分片：按渲染后 HTML 长度分片，超限时按比例重分割 delivery-layer.ts — 新增 deliverRendered() 接收预渲染 chunks - HTML + plain text 双通道，解析失败自动降级纯文本 - sendWithRetry 支持 plainFallback 参数 bridge-manager.ts — Claude 回复改用 markdownToTelegramChunks + deliverRendered - 命令响应和错误消息保持 escapeHtml + deliver Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
7418andClaude Opus 4.6 committed
1 parent
8526b58d59
commit
38e7c96daa
8 files changed
+1569
-14
No files matched your search
@@ -15,6 +15,10 @@ src/lib/bridge/
|
||||
├── permission-broker.ts # 权限请求转发到 IM 内联按钮，处理回调审批
|
||||
├── delivery-layer.ts # 出站消息分片、限流、重试退避、HTML 降级
|
||||
├── bridge-manager.ts # 生命周期编排，adapter 事件循环，/stop abort，命令路由
|
||||
├── markdown/
|
||||
│ ├── ir.ts # Markdown → IR 中间表示解析器（基于 markdown-it）
|
||||
│ ├── render.ts # IR → 格式化输出的通用标记渲染器
|
||||
│ └── telegram.ts # Telegram HTML 渲染 + 文件引用保护 + render-first 分片
|
||||
├── adapters/
|
||||
│ ├── index.ts # Adapter 目录文件（side-effect import 自注册所有 adapter）
|
||||
│ ├── telegram-adapter.ts # Telegram 长轮询 + offset 安全水位 + 图片/相册处理 + 自注册
|
||||
@@ -43,7 +47,8 @@ Telegram 消息 → TelegramAdapter.pollLoop()
|
||||
→ text/tool_use/tool_result → 累积内容块
|
||||
→ result → 捕获 tokenUsage + sdkSessionId
|
||||
→ addMessage() 保存到 DB
|
||||
→ DeliveryLayer.deliver() → 分片 + 限流 + 发送到 Telegram
|
||||
→ markdownToTelegramChunks() → Markdown→IR→HTML render-first 分片
|
||||
→ DeliveryLayer.deliverRendered() → 限流 + HTML/plain 双通道发送到 Telegram
|
||||
→ finally: adapter.acknowledgeUpdate(updateId) → 推进 committedOffset 并持久化
|
||||
```
|
||||
|
||||
@@ -93,6 +98,15 @@ PermissionBroker 在处理 IM 内联按钮回调时，验证 callbackData 中的
|
||||
**10. 图片消息 DB 格式统一**
|
||||
Bridge 和桌面端使用相同的消息存储格式：图片写入 `.codepilot-uploads/`，消息 content 以 `<!--files:[{id,name,type,size,filePath}]-->text` 格式保存。桌面 UI 的 `MessageItem.parseMessageFiles()` 解析后通过 `FileAttachmentDisplay` + `/api/uploads?path=` 渲染缩略图。`conversation-engine.ts` 中 `getSession()` 提前到文件持久化之前调用，确保 workingDirectory 可用。
|
||||
|
||||
**11. Telegram 出站 Markdown 渲染**
|
||||
Claude 的回复是 Markdown 格式，Telegram 仅支持有限 HTML 标签（b/i/s/code/pre+code/blockquote/a）。采用三层架构将 Markdown 转换为 Telegram HTML：
|
||||
|
||||
- **IR 层**（`markdown/ir.ts`）：使用 markdown-it 将 Markdown 解析为中间表示 `MarkdownIR = { text, styles[], links[] }`。text 是纯文本，styles 是 `{ start, end, style }` 区间标记。支持 bold/italic/strikethrough/code/code_block/blockquote/links/lists/headings/tables/hr。表格使用 code-block 模式渲染为 ASCII 表格（包裹在 `<pre><code>` 中保留对齐）。HTML 内联标签中的 `<br>` 被转换为换行符。
|
||||
- **渲染层**（`markdown/render.ts`）：通用标记渲染器 `renderMarkdownWithMarkers(ir, options)`，接受样式→标签映射表 + escapeText + buildLink 回调，输出格式化文本。使用 boundary tracking + LIFO stack 处理嵌套。
|
||||
- **Telegram 层**（`markdown/telegram.ts`）：组合 IR+渲染器，映射样式到 Telegram HTML 标签。`wrapFileReferencesInHtml()` 防止 `README.md`、`main.go` 等文件名被 Telegram linkify 误识别为 URL（用 `<code>` 包裹）。`markdownToTelegramChunks(text, limit)` 实现 render-first 分片：先按 IR text 长度分块，再渲染每块为 HTML，若 HTML 超出 4096 限制则按比例重新分割。
|
||||
|
||||
`bridge-manager.ts` 对 Claude 回复调用 `markdownToTelegramChunks()`，通过 `deliverRendered()` 发送预渲染 chunks（每个 chunk 包含 html + text 双通道，HTML 解析失败自动降级纯文本）。命令响应和错误消息仍使用 `escapeHtml()` + `deliver()`。
|
||||
|
||||
## 设置项（settings 表）
|
||||
|
||||
| Key | 说明 |
|
||||
|
||||
Generated
+88
-2
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "codepilot",
|
||||
"version": "0.21.2",
|
||||
"version": "0.22.0",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "codepilot",
|
||||
"version": "0.21.2",
|
||||
"version": "0.22.0",
|
||||
"dependencies": {
|
||||
"@ai-sdk/anthropic": "^3.0.47",
|
||||
"@ai-sdk/google": "^3.0.31",
|
||||
@@ -29,6 +29,7 @@
|
||||
"cmdk": "^1.1.1",
|
||||
"electron-updater": "^6.8.3",
|
||||
"lucide-react": "^0.563.0",
|
||||
"markdown-it": "^14.1.1",
|
||||
"motion": "^12.33.0",
|
||||
"nanoid": "^5.1.6",
|
||||
"next": "16.1.6",
|
||||
@@ -51,6 +52,7 @@
|
||||
"@playwright/test": "^1.58.1",
|
||||
"@tailwindcss/postcss": "^4",
|
||||
"@types/better-sqlite3": "^7.6.13",
|
||||
"@types/markdown-it": "^14.1.2",
|
||||
"@types/node": "^20",
|
||||
"@types/react": "^19",
|
||||
"@types/react-dom": "^19",
|
||||
@@ -6953,6 +6955,24 @@
|
||||
"@types/node": "*"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/linkify-it": {
|
||||
"version": "5.0.0",
|
||||
"resolved": "https://registry.npmjs.org/@types/linkify-it/-/linkify-it-5.0.0.tgz",
|
||||
"integrity": "sha512-sVDA58zAw4eWAffKOaQH5/5j3XeayukzDk+ewSsnv3p4yJEZHCCzMDiZM8e0OUrRvmpGZ85jf4yDHkHsgBNr9Q==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@types/markdown-it": {
|
||||
"version": "14.1.2",
|
||||
"resolved": "https://registry.npmjs.org/@types/markdown-it/-/markdown-it-14.1.2.tgz",
|
||||
"integrity": "sha512-promo4eFwuiW+TfGxhi+0x3czqTYJkG8qB17ZUJiVF10Xm7NLVRSLUsfRTU/6h1e24VvRnXCx+hG7li58lkzog==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@types/linkify-it": "^5",
|
||||
"@types/mdurl": "^2"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/mdast": {
|
||||
"version": "4.0.4",
|
||||
"resolved": "https://registry.npmjs.org/@types/mdast/-/mdast-4.0.4.tgz",
|
||||
@@ -6962,6 +6982,13 @@
|
||||
"@types/unist": "*"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/mdurl": {
|
||||
"version": "2.0.0",
|
||||
"resolved": "https://registry.npmjs.org/@types/mdurl/-/mdurl-2.0.0.tgz",
|
||||
"integrity": "sha512-RGdgjQUZba5p6QEFAVx2OGb8rQDL/cPRG7GiedRzMcJ1tYnUANBncjbSB1NRGwbvjcPeikRABz2nshyPk1bhWg==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@types/mdx": {
|
||||
"version": "2.0.13",
|
||||
"resolved": "https://registry.npmjs.org/@types/mdx/-/mdx-2.0.13.tgz",
|
||||
@@ -14752,6 +14779,50 @@
|
||||
"url": "https://github.com/sponsors/sindresorhus"
|
||||
}
|
||||
},
|
||||
"node_modules/markdown-it": {
|
||||
"version": "14.1.1",
|
||||
"resolved": "https://registry.npmjs.org/markdown-it/-/markdown-it-14.1.1.tgz",
|
||||
"integrity": "sha512-BuU2qnTti9YKgK5N+IeMubp14ZUKUUw7yeJbkjtosvHiP0AZ5c8IAgEMk79D0eC8F23r4Ac/q8cAIFdm2FtyoA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"argparse": "^2.0.1",
|
||||
"entities": "^4.4.0",
|
||||
"linkify-it": "^5.0.0",
|
||||
"mdurl": "^2.0.0",
|
||||
"punycode.js": "^2.3.1",
|
||||
"uc.micro": "^2.1.0"
|
||||
},
|
||||
"bin": {
|
||||
"markdown-it": "bin/markdown-it.mjs"
|
||||
}
|
||||
},
|
||||
"node_modules/markdown-it/node_modules/entities": {
|
||||
"version": "4.5.0",
|
||||
"resolved": "https://registry.npmjs.org/entities/-/entities-4.5.0.tgz",
|
||||
"integrity": "sha512-V0hjH4dGPh9Ao5p0MoRY6BVqtwCjhz6vI5LT8AJ55H+4g9/4vbHx1I54fS0XuclLhDHArPQCiMjDxjaL8fPxhw==",
|
||||
"license": "BSD-2-Clause",
|
||||
"engines": {
|
||||
"node": ">=0.12"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/fb55/entities?sponsor=1"
|
||||
}
|
||||
},
|
||||
"node_modules/markdown-it/node_modules/linkify-it": {
|
||||
"version": "5.0.0",
|
||||
"resolved": "https://registry.npmjs.org/linkify-it/-/linkify-it-5.0.0.tgz",
|
||||
"integrity": "sha512-5aHCbzQRADcdP+ATqnDuhhJ/MRIqDkZX5pyjFHRRysS8vZ5AbqGEoFIb6pYHPZ+L/OC2Lc+xT8uHVVR5CAK/wQ==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"uc.micro": "^2.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/markdown-it/node_modules/uc.micro": {
|
||||
"version": "2.1.0",
|
||||
"resolved": "https://registry.npmjs.org/uc.micro/-/uc.micro-2.1.0.tgz",
|
||||
"integrity": "sha512-ARDJmphmdvUk6Glw7y9DQ2bFkKBHwQHLi2lsaH6PPmz/Ka9sFOBsBluozhDltWmnv9u/cF6Rt87znRTPV+yp/A==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/markdown-table": {
|
||||
"version": "3.0.4",
|
||||
"resolved": "https://registry.npmjs.org/markdown-table/-/markdown-table-3.0.4.tgz",
|
||||
@@ -15132,6 +15203,12 @@
|
||||
"url": "https://opencollective.com/unified"
|
||||
}
|
||||
},
|
||||
"node_modules/mdurl": {
|
||||
"version": "2.0.0",
|
||||
"resolved": "https://registry.npmjs.org/mdurl/-/mdurl-2.0.0.tgz",
|
||||
"integrity": "sha512-Lf+9+2r+Tdp5wXDXC4PcIBjTDtq4UKjCPMQhKIuzpJNW0b96kVqSwW0bT7FhRSfmAiFYgP+SCRvdrDozfh0U5w==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/merge-value": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/merge-value/-/merge-value-1.0.0.tgz",
|
||||
@@ -17570,6 +17647,15 @@
|
||||
"node": ">=6"
|
||||
}
|
||||
},
|
||||
"node_modules/punycode.js": {
|
||||
"version": "2.3.1",
|
||||
"resolved": "https://registry.npmjs.org/punycode.js/-/punycode.js-2.3.1.tgz",
|
||||
"integrity": "sha512-uxFIHU0YlHYhDQtV4R9J6a52SLx28BCjT+4ieh7IGbgwVJWO+km431c4yRlREUAsAmt/uMjQUyQHNEPf0M39CA==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=6"
|
||||
}
|
||||
},
|
||||
"node_modules/query-string": {
|
||||
"version": "9.3.1",
|
||||
"resolved": "https://registry.npmjs.org/query-string/-/query-string-9.3.1.tgz",
|
||||
|
||||
+3
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "codepilot",
|
||||
"version": "0.21.2",
|
||||
"version": "0.22.0",
|
||||
"private": true,
|
||||
"author": {
|
||||
"name": "op7418",
|
||||
@@ -42,6 +42,7 @@
|
||||
"cmdk": "^1.1.1",
|
||||
"electron-updater": "^6.8.3",
|
||||
"lucide-react": "^0.563.0",
|
||||
"markdown-it": "^14.1.1",
|
||||
"motion": "^12.33.0",
|
||||
"nanoid": "^5.1.6",
|
||||
"next": "16.1.6",
|
||||
@@ -64,6 +65,7 @@
|
||||
"@playwright/test": "^1.58.1",
|
||||
"@tailwindcss/postcss": "^4",
|
||||
"@types/better-sqlite3": "^7.6.13",
|
||||
"@types/markdown-it": "^14.1.2",
|
||||
"@types/node": "^20",
|
||||
"@types/react": "^19",
|
||||
"@types/react-dom": "^19",
|
||||
|
||||
@@ -15,7 +15,8 @@ import './adapters';
|
||||
import * as router from './channel-router';
|
||||
import * as engine from './conversation-engine';
|
||||
import * as broker from './permission-broker';
|
||||
import { deliver } from './delivery-layer';
|
||||
import { deliver, deliverRendered } from './delivery-layer';
|
||||
import { markdownToTelegramChunks } from './markdown/telegram';
|
||||
import { getSetting, insertAuditLog, updateChannelBinding } from '../db';
|
||||
import { setBridgeModeActive } from '../telegram-bot';
|
||||
import { escapeHtml } from './adapters/telegram-utils';
|
||||
@@ -382,16 +383,14 @@ async function handleMessage(
|
||||
);
|
||||
}, taskAbort.signal, hasAttachments ? msg.attachments : undefined);
|
||||
|
||||
// Send response text
|
||||
// Send response text — render Markdown to Telegram HTML
|
||||
if (result.responseText) {
|
||||
const response: OutboundMessage = {
|
||||
address: msg.address,
|
||||
text: escapeHtml(result.responseText),
|
||||
parseMode: 'HTML',
|
||||
};
|
||||
await deliver(adapter, response, {
|
||||
sessionId: binding.codepilotSessionId,
|
||||
});
|
||||
const chunks = markdownToTelegramChunks(result.responseText, 4096);
|
||||
if (chunks.length > 0) {
|
||||
await deliverRendered(adapter, msg.address, chunks, {
|
||||
sessionId: binding.codepilotSessionId,
|
||||
});
|
||||
}
|
||||
} else if (result.hasError) {
|
||||
const errorResponse: OutboundMessage = {
|
||||
address: msg.address,
|
||||
|
||||
@@ -5,10 +5,12 @@
|
||||
|
||||
import type {
|
||||
ChannelType,
|
||||
ChannelAddress,
|
||||
OutboundMessage,
|
||||
SendResult,
|
||||
PLATFORM_LIMITS,
|
||||
} from './types';
|
||||
import type { TelegramChunk } from './markdown/telegram';
|
||||
import { PLATFORM_LIMITS as limits } from './types';
|
||||
import type { BaseChannelAdapter } from './channel-adapter';
|
||||
import {
|
||||
@@ -222,6 +224,7 @@ export async function deliver(
|
||||
async function sendWithRetry(
|
||||
adapter: BaseChannelAdapter,
|
||||
message: OutboundMessage,
|
||||
plainFallback?: string,
|
||||
): Promise<SendResult> {
|
||||
let lastError: string | undefined;
|
||||
|
||||
@@ -234,8 +237,10 @@ async function sendWithRetry(
|
||||
|
||||
// HTML parse error: immediately fallback to plain text (no retry needed)
|
||||
if (category === 'parse_error' && message.parseMode === 'HTML') {
|
||||
const fallbackText = plainFallback || message.text;
|
||||
const plainResult = await adapter.send({
|
||||
...message,
|
||||
text: fallbackText,
|
||||
parseMode: 'plain',
|
||||
});
|
||||
if (plainResult.ok) return plainResult;
|
||||
@@ -260,3 +265,73 @@ async function sendWithRetry(
|
||||
|
||||
return { ok: false, error: lastError || 'Max retries exceeded' };
|
||||
}
|
||||
|
||||
/**
|
||||
* Deliver pre-rendered chunks (from Markdown renderer).
|
||||
* Each chunk already has HTML and plain text fallback.
|
||||
*/
|
||||
export async function deliverRendered(
|
||||
adapter: BaseChannelAdapter,
|
||||
address: ChannelAddress,
|
||||
chunks: TelegramChunk[],
|
||||
opts?: { sessionId?: string; dedupKey?: string },
|
||||
): Promise<SendResult> {
|
||||
// Dedup check
|
||||
if (opts?.dedupKey) {
|
||||
if (checkDedup(opts.dedupKey)) {
|
||||
return { ok: true, messageId: undefined };
|
||||
}
|
||||
}
|
||||
if (Math.random() < 0.01) {
|
||||
try { cleanupExpiredDedup(); } catch { /* best effort */ }
|
||||
}
|
||||
|
||||
let lastMessageId: string | undefined;
|
||||
|
||||
for (let i = 0; i < chunks.length; i++) {
|
||||
await rateLimiter.acquire(address.chatId);
|
||||
if (i > 0) {
|
||||
await new Promise(r => setTimeout(r, INTER_CHUNK_DELAY_MS));
|
||||
}
|
||||
|
||||
const chunk = chunks[i];
|
||||
const htmlMessage: OutboundMessage = {
|
||||
address,
|
||||
text: chunk.html,
|
||||
parseMode: 'HTML',
|
||||
};
|
||||
|
||||
// Try HTML first, fall back to plain text on parse error
|
||||
const result = await sendWithRetry(adapter, htmlMessage, chunk.text);
|
||||
if (!result.ok) return result;
|
||||
lastMessageId = result.messageId;
|
||||
|
||||
if (result.messageId && opts?.sessionId) {
|
||||
try {
|
||||
insertOutboundRef({
|
||||
channelType: adapter.channelType,
|
||||
chatId: address.chatId,
|
||||
codepilotSessionId: opts.sessionId,
|
||||
platformMessageId: result.messageId,
|
||||
purpose: 'response',
|
||||
});
|
||||
} catch { /* best effort */ }
|
||||
}
|
||||
}
|
||||
|
||||
if (opts?.dedupKey) {
|
||||
try { insertDedup(opts.dedupKey); } catch { /* best effort */ }
|
||||
}
|
||||
|
||||
try {
|
||||
insertAuditLog({
|
||||
channelType: adapter.channelType,
|
||||
chatId: address.chatId,
|
||||
direction: 'outbound',
|
||||
messageId: lastMessageId || '',
|
||||
summary: chunks.map(c => c.text).join('').slice(0, 200),
|
||||
});
|
||||
} catch { /* best effort */ }
|
||||
|
||||
return { ok: true, messageId: lastMessageId };
|
||||
}
|
||||
@@ -0,0 +1,829 @@
|
||||
import MarkdownIt from 'markdown-it';
|
||||
|
||||
type ListState = {
|
||||
type: 'bullet' | 'ordered';
|
||||
index: number;
|
||||
};
|
||||
|
||||
type LinkState = {
|
||||
href: string;
|
||||
labelStart: number;
|
||||
};
|
||||
|
||||
type RenderEnv = {
|
||||
listStack: ListState[];
|
||||
};
|
||||
|
||||
type MarkdownToken = {
|
||||
type: string;
|
||||
content?: string;
|
||||
children?: MarkdownToken[];
|
||||
attrs?: [string, string][];
|
||||
attrGet?: (name: string) => string | null;
|
||||
};
|
||||
|
||||
export type MarkdownStyle =
|
||||
| 'bold'
|
||||
| 'italic'
|
||||
| 'strikethrough'
|
||||
| 'code'
|
||||
| 'code_block'
|
||||
| 'blockquote';
|
||||
|
||||
export type MarkdownStyleSpan = {
|
||||
start: number;
|
||||
end: number;
|
||||
style: MarkdownStyle;
|
||||
};
|
||||
|
||||
export type MarkdownLinkSpan = {
|
||||
start: number;
|
||||
end: number;
|
||||
href: string;
|
||||
};
|
||||
|
||||
export type MarkdownIR = {
|
||||
text: string;
|
||||
styles: MarkdownStyleSpan[];
|
||||
links: MarkdownLinkSpan[];
|
||||
};
|
||||
|
||||
type OpenStyle = {
|
||||
style: MarkdownStyle;
|
||||
start: number;
|
||||
};
|
||||
|
||||
type RenderTarget = {
|
||||
text: string;
|
||||
styles: MarkdownStyleSpan[];
|
||||
openStyles: OpenStyle[];
|
||||
links: MarkdownLinkSpan[];
|
||||
linkStack: LinkState[];
|
||||
};
|
||||
|
||||
type TableCell = {
|
||||
text: string;
|
||||
styles: MarkdownStyleSpan[];
|
||||
links: MarkdownLinkSpan[];
|
||||
};
|
||||
|
||||
type TableState = {
|
||||
headers: TableCell[];
|
||||
rows: TableCell[][];
|
||||
currentRow: TableCell[];
|
||||
currentCell: RenderTarget | null;
|
||||
inHeader: boolean;
|
||||
};
|
||||
|
||||
type RenderState = RenderTarget & {
|
||||
env: RenderEnv;
|
||||
headingStyle: 'none' | 'bold';
|
||||
blockquotePrefix: string;
|
||||
table: TableState | null;
|
||||
};
|
||||
|
||||
export type MarkdownParseOptions = {
|
||||
linkify?: boolean;
|
||||
headingStyle?: 'none' | 'bold';
|
||||
blockquotePrefix?: string;
|
||||
autolink?: boolean;
|
||||
enableTables?: boolean;
|
||||
};
|
||||
|
||||
function createMarkdownIt(options: MarkdownParseOptions): MarkdownIt {
|
||||
const md = new MarkdownIt({
|
||||
html: false,
|
||||
linkify: options.linkify ?? true,
|
||||
breaks: false,
|
||||
typographer: false,
|
||||
});
|
||||
md.enable('strikethrough');
|
||||
if (options.enableTables) {
|
||||
md.enable('table');
|
||||
} else {
|
||||
md.disable('table');
|
||||
}
|
||||
if (options.autolink === false) {
|
||||
md.disable('autolink');
|
||||
}
|
||||
return md;
|
||||
}
|
||||
|
||||
function getAttr(token: MarkdownToken, name: string): string | null {
|
||||
if (token.attrGet) {
|
||||
return token.attrGet(name);
|
||||
}
|
||||
if (token.attrs) {
|
||||
for (const [key, value] of token.attrs) {
|
||||
if (key === name) {
|
||||
return value;
|
||||
}
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function initRenderTarget(): RenderTarget {
|
||||
return {
|
||||
text: '',
|
||||
styles: [],
|
||||
openStyles: [],
|
||||
links: [],
|
||||
linkStack: [],
|
||||
};
|
||||
}
|
||||
|
||||
function resolveRenderTarget(state: RenderState): RenderTarget {
|
||||
return state.table?.currentCell ?? state;
|
||||
}
|
||||
|
||||
function appendText(state: RenderState, value: string) {
|
||||
if (!value) {
|
||||
return;
|
||||
}
|
||||
const target = resolveRenderTarget(state);
|
||||
target.text += value;
|
||||
}
|
||||
|
||||
function openStyle(state: RenderState, style: MarkdownStyle) {
|
||||
const target = resolveRenderTarget(state);
|
||||
target.openStyles.push({ style, start: target.text.length });
|
||||
}
|
||||
|
||||
function closeStyle(state: RenderState, style: MarkdownStyle) {
|
||||
const target = resolveRenderTarget(state);
|
||||
for (let i = target.openStyles.length - 1; i >= 0; i -= 1) {
|
||||
if (target.openStyles[i]?.style === style) {
|
||||
const start = target.openStyles[i].start;
|
||||
target.openStyles.splice(i, 1);
|
||||
const end = target.text.length;
|
||||
if (end > start) {
|
||||
target.styles.push({ start, end, style });
|
||||
}
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function appendParagraphSeparator(state: RenderState) {
|
||||
if (state.env.listStack.length > 0) {
|
||||
return;
|
||||
}
|
||||
if (state.table) {
|
||||
return;
|
||||
}
|
||||
state.text += '\n\n';
|
||||
}
|
||||
|
||||
function appendListPrefix(state: RenderState) {
|
||||
const stack = state.env.listStack;
|
||||
const top = stack[stack.length - 1];
|
||||
if (!top) {
|
||||
return;
|
||||
}
|
||||
top.index += 1;
|
||||
const indent = ' '.repeat(Math.max(0, stack.length - 1));
|
||||
const prefix = top.type === 'ordered' ? `${top.index}. ` : '• ';
|
||||
state.text += `${indent}${prefix}`;
|
||||
}
|
||||
|
||||
function renderInlineCode(state: RenderState, content: string) {
|
||||
if (!content) {
|
||||
return;
|
||||
}
|
||||
const target = resolveRenderTarget(state);
|
||||
const start = target.text.length;
|
||||
target.text += content;
|
||||
target.styles.push({ start, end: start + content.length, style: 'code' });
|
||||
}
|
||||
|
||||
function renderCodeBlock(state: RenderState, content: string) {
|
||||
let code = content ?? '';
|
||||
if (!code.endsWith('\n')) {
|
||||
code = `${code}\n`;
|
||||
}
|
||||
const target = resolveRenderTarget(state);
|
||||
const start = target.text.length;
|
||||
target.text += code;
|
||||
target.styles.push({ start, end: start + code.length, style: 'code_block' });
|
||||
if (state.env.listStack.length === 0) {
|
||||
target.text += '\n';
|
||||
}
|
||||
}
|
||||
|
||||
function handleLinkClose(state: RenderState) {
|
||||
const target = resolveRenderTarget(state);
|
||||
const link = target.linkStack.pop();
|
||||
if (!link?.href) {
|
||||
return;
|
||||
}
|
||||
const href = link.href.trim();
|
||||
if (!href) {
|
||||
return;
|
||||
}
|
||||
const start = link.labelStart;
|
||||
const end = target.text.length;
|
||||
target.links.push({ start, end, href });
|
||||
}
|
||||
|
||||
function initTableState(): TableState {
|
||||
return {
|
||||
headers: [],
|
||||
rows: [],
|
||||
currentRow: [],
|
||||
currentCell: null,
|
||||
inHeader: false,
|
||||
};
|
||||
}
|
||||
|
||||
function finishTableCell(cell: RenderTarget): TableCell {
|
||||
closeRemainingStyles(cell);
|
||||
return {
|
||||
text: cell.text,
|
||||
styles: cell.styles,
|
||||
links: cell.links,
|
||||
};
|
||||
}
|
||||
|
||||
function trimCell(cell: TableCell): TableCell {
|
||||
const text = cell.text;
|
||||
let start = 0;
|
||||
let end = text.length;
|
||||
while (start < end && /\s/.test(text[start] ?? '')) {
|
||||
start += 1;
|
||||
}
|
||||
while (end > start && /\s/.test(text[end - 1] ?? '')) {
|
||||
end -= 1;
|
||||
}
|
||||
if (start === 0 && end === text.length) {
|
||||
return cell;
|
||||
}
|
||||
const trimmedText = text.slice(start, end);
|
||||
const trimmedLength = trimmedText.length;
|
||||
const trimmedStyles: MarkdownStyleSpan[] = [];
|
||||
for (const span of cell.styles) {
|
||||
const sliceStart = Math.max(0, span.start - start);
|
||||
const sliceEnd = Math.min(trimmedLength, span.end - start);
|
||||
if (sliceEnd > sliceStart) {
|
||||
trimmedStyles.push({ start: sliceStart, end: sliceEnd, style: span.style });
|
||||
}
|
||||
}
|
||||
const trimmedLinks: MarkdownLinkSpan[] = [];
|
||||
for (const span of cell.links) {
|
||||
const sliceStart = Math.max(0, span.start - start);
|
||||
const sliceEnd = Math.min(trimmedLength, span.end - start);
|
||||
if (sliceEnd > sliceStart) {
|
||||
trimmedLinks.push({ start: sliceStart, end: sliceEnd, href: span.href });
|
||||
}
|
||||
}
|
||||
return { text: trimmedText, styles: trimmedStyles, links: trimmedLinks };
|
||||
}
|
||||
|
||||
function appendCell(state: RenderState, cell: TableCell) {
|
||||
if (!cell.text) {
|
||||
return;
|
||||
}
|
||||
const start = state.text.length;
|
||||
state.text += cell.text;
|
||||
for (const span of cell.styles) {
|
||||
state.styles.push({
|
||||
start: start + span.start,
|
||||
end: start + span.end,
|
||||
style: span.style,
|
||||
});
|
||||
}
|
||||
for (const link of cell.links) {
|
||||
state.links.push({
|
||||
start: start + link.start,
|
||||
end: start + link.end,
|
||||
href: link.href,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
function renderTableAsBullets(state: RenderState) {
|
||||
if (!state.table) {
|
||||
return;
|
||||
}
|
||||
const headers = state.table.headers.map(trimCell);
|
||||
const rows = state.table.rows.map((row) => row.map(trimCell));
|
||||
|
||||
if (headers.length === 0 && rows.length === 0) {
|
||||
return;
|
||||
}
|
||||
|
||||
const useFirstColAsLabel = headers.length > 1 && rows.length > 0;
|
||||
|
||||
if (useFirstColAsLabel) {
|
||||
for (const row of rows) {
|
||||
if (row.length === 0) {
|
||||
continue;
|
||||
}
|
||||
|
||||
const rowLabel = row[0];
|
||||
if (rowLabel?.text) {
|
||||
const labelStart = state.text.length;
|
||||
appendCell(state, rowLabel);
|
||||
const labelEnd = state.text.length;
|
||||
if (labelEnd > labelStart) {
|
||||
state.styles.push({ start: labelStart, end: labelEnd, style: 'bold' });
|
||||
}
|
||||
state.text += '\n';
|
||||
}
|
||||
|
||||
for (let i = 1; i < row.length; i++) {
|
||||
const header = headers[i];
|
||||
const value = row[i];
|
||||
if (!value?.text) {
|
||||
continue;
|
||||
}
|
||||
state.text += '• ';
|
||||
if (header?.text) {
|
||||
appendCell(state, header);
|
||||
state.text += ': ';
|
||||
} else {
|
||||
state.text += `Column ${i}: `;
|
||||
}
|
||||
appendCell(state, value);
|
||||
state.text += '\n';
|
||||
}
|
||||
state.text += '\n';
|
||||
}
|
||||
} else {
|
||||
for (const row of rows) {
|
||||
for (let i = 0; i < row.length; i++) {
|
||||
const header = headers[i];
|
||||
const value = row[i];
|
||||
if (!value?.text) {
|
||||
continue;
|
||||
}
|
||||
state.text += '• ';
|
||||
if (header?.text) {
|
||||
appendCell(state, header);
|
||||
state.text += ': ';
|
||||
}
|
||||
appendCell(state, value);
|
||||
state.text += '\n';
|
||||
}
|
||||
state.text += '\n';
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function appendCellTextOnly(state: RenderState, cell: TableCell) {
|
||||
if (!cell.text) {
|
||||
return;
|
||||
}
|
||||
state.text += cell.text;
|
||||
// Do not append styles — used for code blocks where inner styles would overlap
|
||||
}
|
||||
|
||||
function renderTableAsCode(state: RenderState) {
|
||||
if (!state.table) {
|
||||
return;
|
||||
}
|
||||
const headers = state.table.headers.map(trimCell);
|
||||
const rows = state.table.rows.map((row) => row.map(trimCell));
|
||||
|
||||
const columnCount = Math.max(headers.length, ...rows.map((row) => row.length));
|
||||
if (columnCount === 0) {
|
||||
return;
|
||||
}
|
||||
|
||||
const widths = Array.from({ length: columnCount }, () => 0);
|
||||
const updateWidths = (cells: TableCell[]) => {
|
||||
for (let i = 0; i < columnCount; i += 1) {
|
||||
const cell = cells[i];
|
||||
const width = cell?.text.length ?? 0;
|
||||
if (widths[i]! < width) {
|
||||
widths[i] = width;
|
||||
}
|
||||
}
|
||||
};
|
||||
updateWidths(headers);
|
||||
for (const row of rows) {
|
||||
updateWidths(row);
|
||||
}
|
||||
|
||||
const codeStart = state.text.length;
|
||||
|
||||
const appendRow = (cells: TableCell[]) => {
|
||||
state.text += '|';
|
||||
for (let i = 0; i < columnCount; i += 1) {
|
||||
state.text += ' ';
|
||||
const cell = cells[i];
|
||||
if (cell) {
|
||||
appendCellTextOnly(state, cell);
|
||||
}
|
||||
const pad = widths[i]! - (cell?.text.length ?? 0);
|
||||
if (pad > 0) {
|
||||
state.text += ' '.repeat(pad);
|
||||
}
|
||||
state.text += ' |';
|
||||
}
|
||||
state.text += '\n';
|
||||
};
|
||||
|
||||
const appendDivider = () => {
|
||||
state.text += '|';
|
||||
for (let i = 0; i < columnCount; i += 1) {
|
||||
const dashCount = Math.max(3, widths[i]!);
|
||||
state.text += ` ${'-'.repeat(dashCount)} |`;
|
||||
}
|
||||
state.text += '\n';
|
||||
};
|
||||
|
||||
appendRow(headers);
|
||||
appendDivider();
|
||||
for (const row of rows) {
|
||||
appendRow(row);
|
||||
}
|
||||
|
||||
const codeEnd = state.text.length;
|
||||
if (codeEnd > codeStart) {
|
||||
state.styles.push({ start: codeStart, end: codeEnd, style: 'code_block' });
|
||||
}
|
||||
if (state.env.listStack.length === 0) {
|
||||
state.text += '\n';
|
||||
}
|
||||
}
|
||||
|
||||
function renderTokens(tokens: MarkdownToken[], state: RenderState): void {
|
||||
for (const token of tokens) {
|
||||
switch (token.type) {
|
||||
case 'inline':
|
||||
if (token.children) {
|
||||
renderTokens(token.children, state);
|
||||
}
|
||||
break;
|
||||
case 'text':
|
||||
appendText(state, token.content ?? '');
|
||||
break;
|
||||
case 'em_open':
|
||||
openStyle(state, 'italic');
|
||||
break;
|
||||
case 'em_close':
|
||||
closeStyle(state, 'italic');
|
||||
break;
|
||||
case 'strong_open':
|
||||
openStyle(state, 'bold');
|
||||
break;
|
||||
case 'strong_close':
|
||||
closeStyle(state, 'bold');
|
||||
break;
|
||||
case 's_open':
|
||||
openStyle(state, 'strikethrough');
|
||||
break;
|
||||
case 's_close':
|
||||
closeStyle(state, 'strikethrough');
|
||||
break;
|
||||
case 'code_inline':
|
||||
renderInlineCode(state, token.content ?? '');
|
||||
break;
|
||||
case 'link_open': {
|
||||
const href = getAttr(token, 'href') ?? '';
|
||||
const target = resolveRenderTarget(state);
|
||||
target.linkStack.push({ href, labelStart: target.text.length });
|
||||
break;
|
||||
}
|
||||
case 'link_close':
|
||||
handleLinkClose(state);
|
||||
break;
|
||||
case 'image':
|
||||
appendText(state, token.content ?? '');
|
||||
break;
|
||||
case 'softbreak':
|
||||
case 'hardbreak':
|
||||
appendText(state, '\n');
|
||||
break;
|
||||
case 'paragraph_close':
|
||||
appendParagraphSeparator(state);
|
||||
break;
|
||||
case 'heading_open':
|
||||
if (state.headingStyle === 'bold') {
|
||||
openStyle(state, 'bold');
|
||||
}
|
||||
break;
|
||||
case 'heading_close':
|
||||
if (state.headingStyle === 'bold') {
|
||||
closeStyle(state, 'bold');
|
||||
}
|
||||
appendParagraphSeparator(state);
|
||||
break;
|
||||
case 'blockquote_open':
|
||||
if (state.blockquotePrefix) {
|
||||
state.text += state.blockquotePrefix;
|
||||
}
|
||||
openStyle(state, 'blockquote');
|
||||
break;
|
||||
case 'blockquote_close':
|
||||
closeStyle(state, 'blockquote');
|
||||
break;
|
||||
case 'bullet_list_open':
|
||||
if (state.env.listStack.length > 0) {
|
||||
state.text += '\n';
|
||||
}
|
||||
state.env.listStack.push({ type: 'bullet', index: 0 });
|
||||
break;
|
||||
case 'bullet_list_close':
|
||||
state.env.listStack.pop();
|
||||
if (state.env.listStack.length === 0) {
|
||||
state.text += '\n';
|
||||
}
|
||||
break;
|
||||
case 'ordered_list_open': {
|
||||
if (state.env.listStack.length > 0) {
|
||||
state.text += '\n';
|
||||
}
|
||||
const start = Number(getAttr(token, 'start') ?? '1');
|
||||
state.env.listStack.push({ type: 'ordered', index: start - 1 });
|
||||
break;
|
||||
}
|
||||
case 'ordered_list_close':
|
||||
state.env.listStack.pop();
|
||||
if (state.env.listStack.length === 0) {
|
||||
state.text += '\n';
|
||||
}
|
||||
break;
|
||||
case 'list_item_open':
|
||||
appendListPrefix(state);
|
||||
break;
|
||||
case 'list_item_close':
|
||||
if (!state.text.endsWith('\n')) {
|
||||
state.text += '\n';
|
||||
}
|
||||
break;
|
||||
case 'code_block':
|
||||
case 'fence':
|
||||
renderCodeBlock(state, token.content ?? '');
|
||||
break;
|
||||
case 'html_block':
|
||||
case 'html_inline': {
|
||||
const htmlContent = token.content ?? '';
|
||||
// Convert <br> variants to newlines instead of appending raw HTML
|
||||
const converted = htmlContent.replace(/<br\s*\/?>/gi, '\n');
|
||||
appendText(state, converted);
|
||||
break;
|
||||
}
|
||||
|
||||
// Table handling
|
||||
case 'table_open':
|
||||
state.table = initTableState();
|
||||
break;
|
||||
case 'table_close':
|
||||
if (state.table) {
|
||||
renderTableAsCode(state);
|
||||
}
|
||||
state.table = null;
|
||||
break;
|
||||
case 'thead_open':
|
||||
if (state.table) {
|
||||
state.table.inHeader = true;
|
||||
}
|
||||
break;
|
||||
case 'thead_close':
|
||||
if (state.table) {
|
||||
state.table.inHeader = false;
|
||||
}
|
||||
break;
|
||||
case 'tbody_open':
|
||||
case 'tbody_close':
|
||||
break;
|
||||
case 'tr_open':
|
||||
if (state.table) {
|
||||
state.table.currentRow = [];
|
||||
}
|
||||
break;
|
||||
case 'tr_close':
|
||||
if (state.table) {
|
||||
if (state.table.inHeader) {
|
||||
state.table.headers = state.table.currentRow;
|
||||
} else {
|
||||
state.table.rows.push(state.table.currentRow);
|
||||
}
|
||||
state.table.currentRow = [];
|
||||
}
|
||||
break;
|
||||
case 'th_open':
|
||||
case 'td_open':
|
||||
if (state.table) {
|
||||
state.table.currentCell = initRenderTarget();
|
||||
}
|
||||
break;
|
||||
case 'th_close':
|
||||
case 'td_close':
|
||||
if (state.table?.currentCell) {
|
||||
state.table.currentRow.push(finishTableCell(state.table.currentCell));
|
||||
state.table.currentCell = null;
|
||||
}
|
||||
break;
|
||||
|
||||
case 'hr':
|
||||
state.text += '───\n\n';
|
||||
break;
|
||||
default:
|
||||
if (token.children) {
|
||||
renderTokens(token.children, state);
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function closeRemainingStyles(target: RenderTarget) {
|
||||
for (let i = target.openStyles.length - 1; i >= 0; i -= 1) {
|
||||
const open = target.openStyles[i];
|
||||
const end = target.text.length;
|
||||
if (end > open.start) {
|
||||
target.styles.push({
|
||||
start: open.start,
|
||||
end,
|
||||
style: open.style,
|
||||
});
|
||||
}
|
||||
}
|
||||
target.openStyles = [];
|
||||
}
|
||||
|
||||
function clampStyleSpans(spans: MarkdownStyleSpan[], maxLength: number): MarkdownStyleSpan[] {
|
||||
const clamped: MarkdownStyleSpan[] = [];
|
||||
for (const span of spans) {
|
||||
const start = Math.max(0, Math.min(span.start, maxLength));
|
||||
const end = Math.max(start, Math.min(span.end, maxLength));
|
||||
if (end > start) {
|
||||
clamped.push({ start, end, style: span.style });
|
||||
}
|
||||
}
|
||||
return clamped;
|
||||
}
|
||||
|
||||
function clampLinkSpans(spans: MarkdownLinkSpan[], maxLength: number): MarkdownLinkSpan[] {
|
||||
const clamped: MarkdownLinkSpan[] = [];
|
||||
for (const span of spans) {
|
||||
const start = Math.max(0, Math.min(span.start, maxLength));
|
||||
const end = Math.max(start, Math.min(span.end, maxLength));
|
||||
if (end > start) {
|
||||
clamped.push({ start, end, href: span.href });
|
||||
}
|
||||
}
|
||||
return clamped;
|
||||
}
|
||||
|
||||
function mergeStyleSpans(spans: MarkdownStyleSpan[]): MarkdownStyleSpan[] {
|
||||
const sorted = [...spans].toSorted((a, b) => {
|
||||
if (a.start !== b.start) {
|
||||
return a.start - b.start;
|
||||
}
|
||||
if (a.end !== b.end) {
|
||||
return a.end - b.end;
|
||||
}
|
||||
return a.style.localeCompare(b.style);
|
||||
});
|
||||
|
||||
const merged: MarkdownStyleSpan[] = [];
|
||||
for (const span of sorted) {
|
||||
const prev = merged[merged.length - 1];
|
||||
if (
|
||||
prev &&
|
||||
prev.style === span.style &&
|
||||
(span.start < prev.end || (span.start === prev.end && span.style !== 'blockquote'))
|
||||
) {
|
||||
prev.end = Math.max(prev.end, span.end);
|
||||
continue;
|
||||
}
|
||||
merged.push({ ...span });
|
||||
}
|
||||
return merged;
|
||||
}
|
||||
|
||||
function sliceStyleSpans(
|
||||
spans: MarkdownStyleSpan[],
|
||||
start: number,
|
||||
end: number,
|
||||
): MarkdownStyleSpan[] {
|
||||
if (spans.length === 0) {
|
||||
return [];
|
||||
}
|
||||
const sliced: MarkdownStyleSpan[] = [];
|
||||
for (const span of spans) {
|
||||
const sliceStart = Math.max(span.start, start);
|
||||
const sliceEnd = Math.min(span.end, end);
|
||||
if (sliceEnd > sliceStart) {
|
||||
sliced.push({
|
||||
start: sliceStart - start,
|
||||
end: sliceEnd - start,
|
||||
style: span.style,
|
||||
});
|
||||
}
|
||||
}
|
||||
return mergeStyleSpans(sliced);
|
||||
}
|
||||
|
||||
function sliceLinkSpans(spans: MarkdownLinkSpan[], start: number, end: number): MarkdownLinkSpan[] {
|
||||
if (spans.length === 0) {
|
||||
return [];
|
||||
}
|
||||
const sliced: MarkdownLinkSpan[] = [];
|
||||
for (const span of spans) {
|
||||
const sliceStart = Math.max(span.start, start);
|
||||
const sliceEnd = Math.min(span.end, end);
|
||||
if (sliceEnd > sliceStart) {
|
||||
sliced.push({
|
||||
start: sliceStart - start,
|
||||
end: sliceEnd - start,
|
||||
href: span.href,
|
||||
});
|
||||
}
|
||||
}
|
||||
return sliced;
|
||||
}
|
||||
|
||||
function chunkText(text: string, limit: number): string[] {
|
||||
if (text.length <= limit) return [text];
|
||||
const chunks: string[] = [];
|
||||
let remaining = text;
|
||||
while (remaining.length > 0) {
|
||||
if (remaining.length <= limit) { chunks.push(remaining); break; }
|
||||
let splitIdx = remaining.lastIndexOf('\n', limit);
|
||||
if (splitIdx <= 0 || splitIdx < limit * 0.5) splitIdx = limit;
|
||||
chunks.push(remaining.slice(0, splitIdx));
|
||||
remaining = remaining.slice(splitIdx).replace(/^\n/, '');
|
||||
}
|
||||
return chunks;
|
||||
}
|
||||
|
||||
export function markdownToIR(markdown: string, options: MarkdownParseOptions = {}): MarkdownIR {
|
||||
const env: RenderEnv = { listStack: [] };
|
||||
const md = createMarkdownIt(options);
|
||||
const tokens = md.parse(markdown ?? '', env as unknown as object);
|
||||
|
||||
const state: RenderState = {
|
||||
text: '',
|
||||
styles: [],
|
||||
openStyles: [],
|
||||
links: [],
|
||||
linkStack: [],
|
||||
env,
|
||||
headingStyle: options.headingStyle ?? 'none',
|
||||
blockquotePrefix: options.blockquotePrefix ?? '',
|
||||
table: null,
|
||||
};
|
||||
|
||||
renderTokens(tokens as MarkdownToken[], state);
|
||||
closeRemainingStyles(state);
|
||||
|
||||
const trimmedText = state.text.trimEnd();
|
||||
const trimmedLength = trimmedText.length;
|
||||
let codeBlockEnd = 0;
|
||||
for (const span of state.styles) {
|
||||
if (span.style !== 'code_block') {
|
||||
continue;
|
||||
}
|
||||
if (span.end > codeBlockEnd) {
|
||||
codeBlockEnd = span.end;
|
||||
}
|
||||
}
|
||||
const finalLength = Math.max(trimmedLength, codeBlockEnd);
|
||||
const finalText =
|
||||
finalLength === state.text.length ? state.text : state.text.slice(0, finalLength);
|
||||
|
||||
return {
|
||||
text: finalText,
|
||||
styles: mergeStyleSpans(clampStyleSpans(state.styles, finalLength)),
|
||||
links: clampLinkSpans(state.links, finalLength),
|
||||
};
|
||||
}
|
||||
|
||||
export function chunkMarkdownIR(ir: MarkdownIR, limit: number): MarkdownIR[] {
|
||||
if (!ir.text) {
|
||||
return [];
|
||||
}
|
||||
if (limit <= 0 || ir.text.length <= limit) {
|
||||
return [ir];
|
||||
}
|
||||
|
||||
const chunks = chunkText(ir.text, limit);
|
||||
const results: MarkdownIR[] = [];
|
||||
let cursor = 0;
|
||||
|
||||
chunks.forEach((chunk, index) => {
|
||||
if (!chunk) {
|
||||
return;
|
||||
}
|
||||
if (index > 0) {
|
||||
while (cursor < ir.text.length && /\s/.test(ir.text[cursor] ?? '')) {
|
||||
cursor += 1;
|
||||
}
|
||||
}
|
||||
const start = cursor;
|
||||
const end = Math.min(ir.text.length, start + chunk.length);
|
||||
results.push({
|
||||
text: chunk,
|
||||
styles: sliceStyleSpans(ir.styles, start, end),
|
||||
links: sliceLinkSpans(ir.links, start, end),
|
||||
});
|
||||
cursor = end;
|
||||
});
|
||||
|
||||
return results;
|
||||
}
|
||||
@@ -0,0 +1,192 @@
|
||||
import type { MarkdownIR, MarkdownLinkSpan, MarkdownStyle, MarkdownStyleSpan } from './ir';
|
||||
|
||||
export type RenderStyleMarker = {
|
||||
open: string;
|
||||
close: string;
|
||||
};
|
||||
|
||||
export type RenderStyleMap = Partial<Record<MarkdownStyle, RenderStyleMarker>>;
|
||||
|
||||
export type RenderLink = {
|
||||
start: number;
|
||||
end: number;
|
||||
open: string;
|
||||
close: string;
|
||||
};
|
||||
|
||||
export type RenderOptions = {
|
||||
styleMarkers: RenderStyleMap;
|
||||
escapeText: (text: string) => string;
|
||||
buildLink?: (link: MarkdownLinkSpan, text: string) => RenderLink | null;
|
||||
};
|
||||
|
||||
const STYLE_ORDER: MarkdownStyle[] = [
|
||||
'blockquote',
|
||||
'code_block',
|
||||
'code',
|
||||
'bold',
|
||||
'italic',
|
||||
'strikethrough',
|
||||
];
|
||||
|
||||
const STYLE_RANK = new Map<MarkdownStyle, number>(
|
||||
STYLE_ORDER.map((style, index) => [style, index]),
|
||||
);
|
||||
|
||||
function sortStyleSpans(spans: MarkdownStyleSpan[]): MarkdownStyleSpan[] {
|
||||
return [...spans].toSorted((a, b) => {
|
||||
if (a.start !== b.start) {
|
||||
return a.start - b.start;
|
||||
}
|
||||
if (a.end !== b.end) {
|
||||
return b.end - a.end;
|
||||
}
|
||||
return (STYLE_RANK.get(a.style) ?? 0) - (STYLE_RANK.get(b.style) ?? 0);
|
||||
});
|
||||
}
|
||||
|
||||
export function renderMarkdownWithMarkers(ir: MarkdownIR, options: RenderOptions): string {
|
||||
const text = ir.text ?? '';
|
||||
if (!text) {
|
||||
return '';
|
||||
}
|
||||
|
||||
const styleMarkers = options.styleMarkers;
|
||||
const styled = sortStyleSpans(ir.styles.filter((span) => Boolean(styleMarkers[span.style])));
|
||||
|
||||
const boundaries = new Set<number>();
|
||||
boundaries.add(0);
|
||||
boundaries.add(text.length);
|
||||
|
||||
const startsAt = new Map<number, MarkdownStyleSpan[]>();
|
||||
for (const span of styled) {
|
||||
if (span.start === span.end) {
|
||||
continue;
|
||||
}
|
||||
boundaries.add(span.start);
|
||||
boundaries.add(span.end);
|
||||
const bucket = startsAt.get(span.start);
|
||||
if (bucket) {
|
||||
bucket.push(span);
|
||||
} else {
|
||||
startsAt.set(span.start, [span]);
|
||||
}
|
||||
}
|
||||
for (const spans of startsAt.values()) {
|
||||
spans.sort((a, b) => {
|
||||
if (a.end !== b.end) {
|
||||
return b.end - a.end;
|
||||
}
|
||||
return (STYLE_RANK.get(a.style) ?? 0) - (STYLE_RANK.get(b.style) ?? 0);
|
||||
});
|
||||
}
|
||||
|
||||
const linkStarts = new Map<number, RenderLink[]>();
|
||||
if (options.buildLink) {
|
||||
for (const link of ir.links) {
|
||||
if (link.start === link.end) {
|
||||
continue;
|
||||
}
|
||||
const rendered = options.buildLink(link, text);
|
||||
if (!rendered) {
|
||||
continue;
|
||||
}
|
||||
boundaries.add(rendered.start);
|
||||
boundaries.add(rendered.end);
|
||||
const openBucket = linkStarts.get(rendered.start);
|
||||
if (openBucket) {
|
||||
openBucket.push(rendered);
|
||||
} else {
|
||||
linkStarts.set(rendered.start, [rendered]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const points = [...boundaries].toSorted((a, b) => a - b);
|
||||
const stack: { close: string; end: number }[] = [];
|
||||
type OpeningItem =
|
||||
| { end: number; open: string; close: string; kind: 'link'; index: number }
|
||||
| {
|
||||
end: number;
|
||||
open: string;
|
||||
close: string;
|
||||
kind: 'style';
|
||||
style: MarkdownStyle;
|
||||
index: number;
|
||||
};
|
||||
let out = '';
|
||||
|
||||
for (let i = 0; i < points.length; i += 1) {
|
||||
const pos = points[i];
|
||||
|
||||
while (stack.length && stack[stack.length - 1]?.end === pos) {
|
||||
const item = stack.pop();
|
||||
if (item) {
|
||||
out += item.close;
|
||||
}
|
||||
}
|
||||
|
||||
const openingItems: OpeningItem[] = [];
|
||||
|
||||
const openingLinks = linkStarts.get(pos);
|
||||
if (openingLinks && openingLinks.length > 0) {
|
||||
for (const [index, link] of openingLinks.entries()) {
|
||||
openingItems.push({
|
||||
end: link.end,
|
||||
open: link.open,
|
||||
close: link.close,
|
||||
kind: 'link',
|
||||
index,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
const openingStyles = startsAt.get(pos);
|
||||
if (openingStyles) {
|
||||
for (const [index, span] of openingStyles.entries()) {
|
||||
const marker = styleMarkers[span.style];
|
||||
if (!marker) {
|
||||
continue;
|
||||
}
|
||||
openingItems.push({
|
||||
end: span.end,
|
||||
open: marker.open,
|
||||
close: marker.close,
|
||||
kind: 'style',
|
||||
style: span.style,
|
||||
index,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
if (openingItems.length > 0) {
|
||||
openingItems.sort((a, b) => {
|
||||
if (a.end !== b.end) {
|
||||
return b.end - a.end;
|
||||
}
|
||||
if (a.kind !== b.kind) {
|
||||
return a.kind === 'link' ? -1 : 1;
|
||||
}
|
||||
if (a.kind === 'style' && b.kind === 'style') {
|
||||
return (STYLE_RANK.get(a.style) ?? 0) - (STYLE_RANK.get(b.style) ?? 0);
|
||||
}
|
||||
return a.index - b.index;
|
||||
});
|
||||
|
||||
for (const item of openingItems) {
|
||||
out += item.open;
|
||||
stack.push({ close: item.close, end: item.end });
|
||||
}
|
||||
}
|
||||
|
||||
const next = points[i + 1];
|
||||
if (next === undefined) {
|
||||
break;
|
||||
}
|
||||
if (next > pos) {
|
||||
out += options.escapeText(text.slice(pos, next));
|
||||
}
|
||||
}
|
||||
|
||||
return out;
|
||||
}
|
||||
@@ -0,0 +1,358 @@
|
||||
/**
|
||||
* Telegram-specific Markdown renderer.
|
||||
*
|
||||
* Converts Markdown → IR → Telegram-compatible HTML, with file-reference
|
||||
* wrapping and render-first chunking for long messages.
|
||||
*
|
||||
* Ported from openclaw src/telegram/format.ts.
|
||||
*/
|
||||
|
||||
import { markdownToIR, chunkMarkdownIR, type MarkdownIR, type MarkdownLinkSpan } from './ir';
|
||||
import { renderMarkdownWithMarkers } from './render';
|
||||
|
||||
export type TelegramChunk = {
|
||||
html: string;
|
||||
text: string;
|
||||
};
|
||||
|
||||
// ── HTML escaping ─────────────────────────────────────────────
|
||||
|
||||
function escapeHtml(text: string): string {
|
||||
return text.replace(/&/g, '&').replace(/</g, '<').replace(/>/g, '>');
|
||||
}
|
||||
|
||||
function escapeHtmlAttr(text: string): string {
|
||||
return escapeHtml(text).replace(/"/g, '"');
|
||||
}
|
||||
|
||||
// ── File reference detection ──────────────────────────────────
|
||||
|
||||
/**
|
||||
* File extensions that share TLDs and commonly appear in code/documentation.
|
||||
* Wrapped in <code> tags to prevent Telegram from generating
|
||||
* spurious domain registrar previews.
|
||||
*/
|
||||
const FILE_EXTENSIONS_WITH_TLD = new Set([
|
||||
'md', // Markdown (Moldova)
|
||||
'go', // Go language
|
||||
'py', // Python (Paraguay)
|
||||
'pl', // Perl (Poland)
|
||||
'sh', // Shell (Saint Helena)
|
||||
'am', // Automake files (Armenia)
|
||||
'at', // Assembly (Austria)
|
||||
'be', // Backend files (Belgium)
|
||||
'cc', // C++ source (Cocos Islands)
|
||||
]);
|
||||
|
||||
/** Detects when linkify auto-generated a link from a bare filename (e.g. README.md → http://README.md) */
|
||||
function isAutoLinkedFileRef(href: string, label: string): boolean {
|
||||
const stripped = href.replace(/^https?:\/\//i, '');
|
||||
if (stripped !== label) {
|
||||
return false;
|
||||
}
|
||||
const dotIndex = label.lastIndexOf('.');
|
||||
if (dotIndex < 1) {
|
||||
return false;
|
||||
}
|
||||
const ext = label.slice(dotIndex + 1).toLowerCase();
|
||||
if (!FILE_EXTENSIONS_WITH_TLD.has(ext)) {
|
||||
return false;
|
||||
}
|
||||
// Reject if any path segment before the filename contains a dot (looks like a domain)
|
||||
const segments = label.split('/');
|
||||
if (segments.length > 1) {
|
||||
for (let i = 0; i < segments.length - 1; i++) {
|
||||
if (segments[i].includes('.')) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
function buildTelegramLink(link: MarkdownLinkSpan, text: string) {
|
||||
const href = link.href.trim();
|
||||
if (!href) {
|
||||
return null;
|
||||
}
|
||||
if (link.start === link.end) {
|
||||
return null;
|
||||
}
|
||||
// Suppress auto-linkified file references (e.g. README.md → http://README.md)
|
||||
const label = text.slice(link.start, link.end);
|
||||
if (isAutoLinkedFileRef(href, label)) {
|
||||
return null;
|
||||
}
|
||||
const safeHref = escapeHtmlAttr(href);
|
||||
return {
|
||||
start: link.start,
|
||||
end: link.end,
|
||||
open: `<a href="${safeHref}">`,
|
||||
close: '</a>',
|
||||
};
|
||||
}
|
||||
|
||||
// ── Core rendering ────────────────────────────────────────────
|
||||
|
||||
function renderTelegramHtml(ir: MarkdownIR): string {
|
||||
return renderMarkdownWithMarkers(ir, {
|
||||
styleMarkers: {
|
||||
bold: { open: '<b>', close: '</b>' },
|
||||
italic: { open: '<i>', close: '</i>' },
|
||||
strikethrough: { open: '<s>', close: '</s>' },
|
||||
code: { open: '<code>', close: '</code>' },
|
||||
code_block: { open: '<pre><code>', close: '</code></pre>' },
|
||||
blockquote: { open: '<blockquote>', close: '</blockquote>' },
|
||||
},
|
||||
escapeText: escapeHtml,
|
||||
buildLink: buildTelegramLink,
|
||||
});
|
||||
}
|
||||
|
||||
// ── File reference wrapping (post-processing) ─────────────────
|
||||
|
||||
function escapeRegex(str: string): string {
|
||||
return str.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
||||
}
|
||||
|
||||
const FILE_EXTENSIONS_PATTERN = Array.from(FILE_EXTENSIONS_WITH_TLD).map(escapeRegex).join('|');
|
||||
const AUTO_LINKED_ANCHOR_PATTERN = /<a\s+href="https?:\/\/([^"]+)"[^>]*>\1<\/a>/gi;
|
||||
const FILE_REFERENCE_PATTERN = new RegExp(
|
||||
`(^|[^a-zA-Z0-9_\\-/])([a-zA-Z0-9_.\\-./]+\\.(?:${FILE_EXTENSIONS_PATTERN}))(?=$|[^a-zA-Z0-9_\\-/])`,
|
||||
'gi',
|
||||
);
|
||||
const ORPHANED_TLD_PATTERN = new RegExp(
|
||||
`([^a-zA-Z0-9]|^)([A-Za-z]\\.(?:${FILE_EXTENSIONS_PATTERN}))(?=[^a-zA-Z0-9/]|$)`,
|
||||
'g',
|
||||
);
|
||||
const HTML_TAG_PATTERN = /(<\/?)([a-zA-Z][a-zA-Z0-9-]*)\b[^>]*?>/gi;
|
||||
|
||||
function wrapStandaloneFileRef(match: string, prefix: string, filename: string): string {
|
||||
if (filename.startsWith('//')) {
|
||||
return match;
|
||||
}
|
||||
if (/https?:\/\/$/i.test(prefix)) {
|
||||
return match;
|
||||
}
|
||||
return `${prefix}<code>${escapeHtml(filename)}</code>`;
|
||||
}
|
||||
|
||||
function wrapSegmentFileRefs(
|
||||
text: string,
|
||||
codeDepth: number,
|
||||
preDepth: number,
|
||||
anchorDepth: number,
|
||||
): string {
|
||||
if (!text || codeDepth > 0 || preDepth > 0 || anchorDepth > 0) {
|
||||
return text;
|
||||
}
|
||||
const wrappedStandalone = text.replace(FILE_REFERENCE_PATTERN, wrapStandaloneFileRef);
|
||||
return wrappedStandalone.replace(ORPHANED_TLD_PATTERN, (match, prefix: string, tld: string) =>
|
||||
prefix === '>' ? match : `${prefix}<code>${escapeHtml(tld)}</code>`,
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Wraps standalone file references (with TLD extensions) in <code> tags.
|
||||
* Prevents Telegram from treating them as URLs and generating
|
||||
* irrelevant domain registrar previews.
|
||||
*
|
||||
* Runs AFTER markdown→HTML conversion to avoid modifying HTML attributes.
|
||||
* Skips content inside <code>, <pre>, and <a> tags.
|
||||
*/
|
||||
export function wrapFileReferencesInHtml(html: string): string {
|
||||
// Safety-net: de-linkify auto-generated anchors where href="http://<label>"
|
||||
AUTO_LINKED_ANCHOR_PATTERN.lastIndex = 0;
|
||||
const deLinkified = html.replace(AUTO_LINKED_ANCHOR_PATTERN, (_match, label: string) => {
|
||||
if (!isAutoLinkedFileRef(`http://${label}`, label)) {
|
||||
return _match;
|
||||
}
|
||||
return `<code>${escapeHtml(label)}</code>`;
|
||||
});
|
||||
|
||||
// Track nesting depth for tags that should not be modified
|
||||
let codeDepth = 0;
|
||||
let preDepth = 0;
|
||||
let anchorDepth = 0;
|
||||
let result = '';
|
||||
let lastIndex = 0;
|
||||
|
||||
HTML_TAG_PATTERN.lastIndex = 0;
|
||||
let match: RegExpExecArray | null;
|
||||
|
||||
while ((match = HTML_TAG_PATTERN.exec(deLinkified)) !== null) {
|
||||
const tagStart = match.index;
|
||||
const tagEnd = HTML_TAG_PATTERN.lastIndex;
|
||||
const isClosing = match[1] === '</';
|
||||
const tagName = match[2].toLowerCase();
|
||||
|
||||
// Process text before this tag
|
||||
const textBefore = deLinkified.slice(lastIndex, tagStart);
|
||||
result += wrapSegmentFileRefs(textBefore, codeDepth, preDepth, anchorDepth);
|
||||
|
||||
// Update tag depth (clamp at 0 for malformed HTML)
|
||||
if (tagName === 'code') {
|
||||
codeDepth = isClosing ? Math.max(0, codeDepth - 1) : codeDepth + 1;
|
||||
} else if (tagName === 'pre') {
|
||||
preDepth = isClosing ? Math.max(0, preDepth - 1) : preDepth + 1;
|
||||
} else if (tagName === 'a') {
|
||||
anchorDepth = isClosing ? Math.max(0, anchorDepth - 1) : anchorDepth + 1;
|
||||
}
|
||||
|
||||
result += deLinkified.slice(tagStart, tagEnd);
|
||||
lastIndex = tagEnd;
|
||||
}
|
||||
|
||||
// Process remaining text
|
||||
const remainingText = deLinkified.slice(lastIndex);
|
||||
result += wrapSegmentFileRefs(remainingText, codeDepth, preDepth, anchorDepth);
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
// ── IR splitting helpers (local copies for chunk splitting) ───
|
||||
|
||||
function sliceStyleSpans(
|
||||
styles: MarkdownIR['styles'],
|
||||
start: number,
|
||||
end: number,
|
||||
): MarkdownIR['styles'] {
|
||||
return styles.flatMap((span) => {
|
||||
if (span.end <= start || span.start >= end) {
|
||||
return [];
|
||||
}
|
||||
const nextStart = Math.max(span.start, start) - start;
|
||||
const nextEnd = Math.min(span.end, end) - start;
|
||||
if (nextEnd <= nextStart) {
|
||||
return [];
|
||||
}
|
||||
return [{ ...span, start: nextStart, end: nextEnd }];
|
||||
});
|
||||
}
|
||||
|
||||
function sliceLinkSpans(
|
||||
links: MarkdownIR['links'],
|
||||
start: number,
|
||||
end: number,
|
||||
): MarkdownIR['links'] {
|
||||
return links.flatMap((link) => {
|
||||
if (link.end <= start || link.start >= end) {
|
||||
return [];
|
||||
}
|
||||
const nextStart = Math.max(link.start, start) - start;
|
||||
const nextEnd = Math.min(link.end, end) - start;
|
||||
if (nextEnd <= nextStart) {
|
||||
return [];
|
||||
}
|
||||
return [{ ...link, start: nextStart, end: nextEnd }];
|
||||
});
|
||||
}
|
||||
|
||||
function splitMarkdownIRPreserveWhitespace(ir: MarkdownIR, limit: number): MarkdownIR[] {
|
||||
if (!ir.text) {
|
||||
return [];
|
||||
}
|
||||
const normalizedLimit = Math.max(1, Math.floor(limit));
|
||||
if (normalizedLimit <= 0 || ir.text.length <= normalizedLimit) {
|
||||
return [ir];
|
||||
}
|
||||
const chunks: MarkdownIR[] = [];
|
||||
let cursor = 0;
|
||||
while (cursor < ir.text.length) {
|
||||
const end = Math.min(ir.text.length, cursor + normalizedLimit);
|
||||
chunks.push({
|
||||
text: ir.text.slice(cursor, end),
|
||||
styles: sliceStyleSpans(ir.styles, cursor, end),
|
||||
links: sliceLinkSpans(ir.links, cursor, end),
|
||||
});
|
||||
cursor = end;
|
||||
}
|
||||
return chunks;
|
||||
}
|
||||
|
||||
// ── Render-first chunking ─────────────────────────────────────
|
||||
|
||||
function splitTelegramChunkByHtmlLimit(
|
||||
chunk: MarkdownIR,
|
||||
htmlLimit: number,
|
||||
renderedHtmlLength: number,
|
||||
): MarkdownIR[] {
|
||||
const currentTextLength = chunk.text.length;
|
||||
if (currentTextLength <= 1) {
|
||||
return [chunk];
|
||||
}
|
||||
const proportionalLimit = Math.floor(
|
||||
(currentTextLength * htmlLimit) / Math.max(renderedHtmlLength, 1),
|
||||
);
|
||||
const candidateLimit = Math.min(currentTextLength - 1, proportionalLimit);
|
||||
const splitLimit =
|
||||
Number.isFinite(candidateLimit) && candidateLimit > 0
|
||||
? candidateLimit
|
||||
: Math.max(1, Math.floor(currentTextLength / 2));
|
||||
const split = splitMarkdownIRPreserveWhitespace(chunk, splitLimit);
|
||||
if (split.length > 1) {
|
||||
return split;
|
||||
}
|
||||
return splitMarkdownIRPreserveWhitespace(chunk, Math.max(1, Math.floor(currentTextLength / 2)));
|
||||
}
|
||||
|
||||
function renderTelegramChunksWithinHtmlLimit(
|
||||
ir: MarkdownIR,
|
||||
limit: number,
|
||||
): TelegramChunk[] {
|
||||
const normalizedLimit = Math.max(1, Math.floor(limit));
|
||||
const pending = chunkMarkdownIR(ir, normalizedLimit);
|
||||
const rendered: TelegramChunk[] = [];
|
||||
while (pending.length > 0) {
|
||||
const chunk = pending.shift();
|
||||
if (!chunk) {
|
||||
continue;
|
||||
}
|
||||
const html = wrapFileReferencesInHtml(renderTelegramHtml(chunk));
|
||||
if (html.length <= normalizedLimit || chunk.text.length <= 1) {
|
||||
rendered.push({ html, text: chunk.text });
|
||||
continue;
|
||||
}
|
||||
const split = splitTelegramChunkByHtmlLimit(chunk, normalizedLimit, html.length);
|
||||
if (split.length <= 1) {
|
||||
// Worst-case safety: avoid retry loops, deliver the chunk as-is.
|
||||
rendered.push({ html, text: chunk.text });
|
||||
continue;
|
||||
}
|
||||
pending.unshift(...split);
|
||||
}
|
||||
return rendered;
|
||||
}
|
||||
|
||||
// ── Public API ────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Full pipeline: markdown → IR → Telegram HTML with file ref wrapping.
|
||||
*/
|
||||
export function markdownToTelegramHtml(markdown: string): string {
|
||||
const ir = markdownToIR(markdown ?? '', {
|
||||
linkify: true,
|
||||
headingStyle: 'bold',
|
||||
blockquotePrefix: '',
|
||||
enableTables: true,
|
||||
});
|
||||
const html = renderTelegramHtml(ir);
|
||||
return wrapFileReferencesInHtml(html);
|
||||
}
|
||||
|
||||
/**
|
||||
* Render-first chunking: markdown → IR → chunk by IR text →
|
||||
* render each chunk → re-split if HTML exceeds limit.
|
||||
*/
|
||||
export function markdownToTelegramChunks(
|
||||
markdown: string,
|
||||
limit: number,
|
||||
): TelegramChunk[] {
|
||||
const ir = markdownToIR(markdown ?? '', {
|
||||
linkify: true,
|
||||
headingStyle: 'bold',
|
||||
blockquotePrefix: '',
|
||||
enableTables: true,
|
||||
});
|
||||
return renderTelegramChunksWithinHtmlLimit(ir, limit);
|
||||
}
|
||||
Reference in new issue
Block a user