feat(extension): support Hacker News main-post clipping (#505)

Co-authored-by: jingwen.lqh <jingwen.lqh@dtyunxi.com>
This commit is contained in:
LuoQianhongandjingwen.lqh authored and GitHub committed 2026-10-09 12:52:08 +08:00
1 parent 5fe8d75ee4
commit cc18b4329f
25 files changed
+946 -33

No files matched your search

+5 -2
View File
@@ -3,7 +3,7 @@
## Listing
- Name: EdgeEver Web Clipper
- Summary: Save a webpage, selected text, an image, an X post, a Reddit post, a Xiaohongshu note, a Zhihu answer or article, or a GitHub repository to your EdgeEver instance.
- Summary: Save webpages, selections, images, and Hacker News posts to your self-hosted EdgeEver instance.
- Category: Bookmarks
- Homepage: https://edgeever.org
- Support: https://github.com/tianma-if/edgeever/issues
@@ -20,6 +20,7 @@ EdgeEver Web Clipper saves the current webpage, selected text, a right-clicked i
- On Zhihu, right-click the answer or article text and save the title, author, text, and photos. Comments are left out. On a feed or question page, the item under the pointer is saved.
- On Reddit, right-click a post and save its title, author, community, text, links, and images. Comments are left out.
- On a GitHub repository page or code tree, right-click the page background, the description, or the README text and save the repository address and description.
- On Hacker News post detail pages, save the title, author, original time, points at capture, linked page and main-post text. Comments are left out.
- On a YouTube or Bilibili watch page, save that video as a note.
- These commands stay on the top-level right-click menu.
- Convert captured HTML to Markdown locally.
@@ -52,7 +53,8 @@ The project maintainers do not receive or retain this data. Instance settings ar
11. Open one Zhihu answer or article, right-click the text (not a photo), choose **Save answer or article to EdgeEver**, and verify the note contains the title, text, and source link.
12. Open one Reddit post, right-click the post text, choose **Save Reddit post to EdgeEver**, and verify the note contains the title, author, community, and source link without comments.
13. Verify the created notes in the review EdgeEver instance.
14. On a YouTube or Bilibili watch page, right-click the page and choose **Save video note to EdgeEver**. The new menu runs only after that click. Lives and other unsupported pages show a message and create no note.
14. Open a Hacker News post detail page and choose **Save Hacker News post to EdgeEver** or **Clip current page**. Verify the main post and source are saved without comments; a selected passage takes priority for the toolbar action.
15. On a YouTube or Bilibili watch page, right-click the page and choose **Save video note to EdgeEver**. The new menu runs only after that click. Lives and other unsupported pages show a message and create no note.
Restricted browser pages, extension stores, built-in PDF viewers, and other privileged pages cannot be captured.
@@ -84,6 +86,7 @@ EdgeEver 网页剪藏插件可将当前网页、选中的文字、右键选中
- 在小红书笔记正文上右键,保存标题、正文和图片。评论不会写入。
- 在知乎回答或文章正文上右键,保存标题、作者、正文和图片。评论不会写入。首页和问题页保存的是指针下的那一篇。
- 在 Reddit 帖子正文上右键,保存标题、作者、社区、正文、链接和图片,不写入评论。
- 在 Hacker News 主帖详情页保存标题、作者、原始时间、剪藏时的分数、外部链接和主帖正文,不写入评论。
- 可以把 YouTube 或哔哩哔哩播放页保存为视频笔记。
- 这些命令直接出现在右键菜单的第一级。
- 在浏览器本地将 HTML 转换为 Markdown。
+3
View File
@@ -16,12 +16,15 @@ Chrome, Edge, and Firefox Manifest V3 extension for saving the current webpage,
- On Zhihu, right-click the answer or article text and choose **Save Zhihu content to EdgeEver**. The title, author, full text, and photos go into one note. Comments are left out. Right-clicking a photo saves that image instead. An open answer or article can be saved immediately. On the home feed or a question page, the extension remembers the item under the pointer; if that listener is not running yet, right-click the same text once more.
- On Reddit, right-click a post or its title link and choose **Save Reddit post to EdgeEver**. The selected post becomes one note with its title, author, community, post text, source link, linked page, and available images. Comments are left out. The extension reads the selected post through Reddit's JSON response when available and falls back to its visible content. If the pointer listener is unavailable on a feed, right-click the same post once more.
- On a GitHub repository page or code tree, right-click the page background, the description, or the README text and choose **Save this repository to EdgeEver**. The note keeps the repository address, the About text, and the homepage, language, license, and topics when the page shows them. An empty About falls back to the README's first paragraph. Issue, discussion, and single-file pages are left unsaved. The toolbar action uses the same card on those repository pages. Right-clicking a link or an image keeps the existing selection and image commands, so this one stays on the top-level menu.
- On a Hacker News post detail page, right-click the page and choose **Save Hacker News post to EdgeEver**, or click **Clip current page**. The note keeps the title, author, original time, points at capture, external link and main-post text when available, without comments. Ask HN, Show HN and link-only posts are supported. Selecting text before the toolbar action saves only that selection.
- On a YouTube or Bilibili watch page, right-click the page or the player and choose **Save video note to EdgeEver**. The note keeps the source and timestamped captions. A cover is uploaded only when the page provided the image bytes. A summary is added when the token includes `ai:generate` and the workspace has a default model. The command runs only after that click. Lives, premieres, bangumi, and other pages are left unsaved.
- When the page cannot hand over the image bytes, the extension asks once for access to that image's site. After that, later images from the same site save directly.
- Create a searchable EdgeEver memo with the source URL and a `web-clip` tag.
The extension does not use a central relay service. The page content is sent directly to the EdgeEver instance configured by the user.
HN usage and verification limits are documented in [English](../../docs/browser-clipper-hacker-news.md) and [简体中文](../../docs/browser-clipper-hacker-news.zh-CN.md).
## Localization
The extension uses the cross-browser `chrome.i18n` compatibility namespace. English is the fallback locale. Simplified Chinese, Japanese, and Polish are available for browsers whose UI language matches.
+8 -6
View File
@@ -16,7 +16,7 @@
Select the matching language in the Chrome Web Store developer dashboard and enter the corresponding copy below. Store listing localization is separate from the extension's packaged `_locales` messages.
The detailed descriptions below keep the 0.1.9 wording and add one video-note line for extension 0.1.17. The uploaded store package is still 0.1.9 until that version is submitted. An earlier draft stacked every site name in the opening sentence and repeated those names in the closing paragraph. Chrome Web Store rejected it immediately as excessive keywords (violation reference Yellow Argon). Keep each site in one feature line. The Japanese listing uses the English detailed description. The short summary still comes from the packaged `_locales` `extensionDescription`.
The detailed descriptions below include video notes and Hacker News capture prepared for the next store submission. The uploaded store package is still 0.1.9 until that version is submitted. An earlier draft stacked every site name in the opening sentence and repeated those names in the closing paragraph. Chrome Web Store rejected it immediately as excessive keywords (violation reference Yellow Argon). Keep each site in one feature line. The Japanese listing uses the English detailed description. The short summary still comes from the packaged `_locales` `extensionDescription`.
## Upload files
@@ -29,7 +29,7 @@ The detailed descriptions below keep the 0.1.9 wording and add one video-note li
#### Summary
将当前网页、选中文字、右键图片、X 推文、Reddit 帖子、小红书笔记、知乎回答或文章,或 GitHub 仓库保存到你自托管的 EdgeEver 实例。
将网页、选中文字、图片及 Hacker News 主帖保存到你自托管的 EdgeEver 实例。
#### Detailed description
@@ -45,6 +45,7 @@ EdgeEver Web Clipper 可以把当前网页,或你选中的那一部分,保
- 在知乎上保存指针下的回答或文章,包括标题、作者、正文和图片,不写入评论。
- 在 Reddit 上右键帖子,保存标题、作者、社区、正文、链接和图片,不写入评论。
- 在 GitHub 仓库页上保存地址和简介;页面上有主页、语言、许可证和话题时一并写入。
- 在 Hacker News 主帖详情页保存标题、作者、原始时间、剪藏时的分数、外部链接和主帖正文,不写入评论。
- 可以把 YouTube 或哔哩哔哩播放页保存为视频笔记。
- 这些命令直接出现在右键菜单的第一级。
- 笔记会保留来源和剪藏时间。
@@ -59,7 +60,7 @@ EdgeEver 是开源、自托管的现代笔记工作区。项目主页与源代
#### Summary
Save a webpage, selected text, a right-clicked image, an X post, a Reddit post, a Xiaohongshu note, a Zhihu answer or article, or a GitHub repository to your self-hosted EdgeEver.
Save webpages, selections, images, and Hacker News posts to your self-hosted EdgeEver instance.
#### Detailed description
@@ -75,6 +76,7 @@ Key features:
- On Zhihu, save the answer or article under the pointer, including its title, author, text, and photos. Comments are left out.
- On Reddit, right-click a post to save its title, author, community, text, links, and images. Comments are left out.
- On a GitHub repository page, save the address and description, plus the homepage, language, license, and topics when the page shows them.
- On Hacker News post detail pages, save the title, author, original time, points at capture, linked page and main-post text. Comments are left out.
- Save a YouTube or Bilibili watch page as a video note.
- These commands stay on the top-level right-click menu.
- The note keeps its source and the time it was clipped.
@@ -89,12 +91,12 @@ EdgeEver is an open-source, self-hosted modern notes workspace. Project homepage
### Single purpose
Save the current webpage, user-selected text, a user-chosen image, one X post, one Reddit post, one Xiaohongshu note, one Zhihu answer or article, one GitHub repository card, or one video note from a supported watch page to the self-hosted EdgeEver instance explicitly configured by the user.
Save the current webpage, user-selected text, a user-chosen image, one X post, one Reddit post, one Xiaohongshu note, one Zhihu answer or article, one GitHub repository card, one Hacker News post, or one video note from a supported watch page to the self-hosted EdgeEver instance explicitly configured by the user.
### Permission justifications
- `activeTab`: Read the active page only after the user clicks the extension's save action or chooses a save command for selected text, an image, an X post, a Reddit post, a Xiaohongshu note, a Zhihu item, a GitHub repository, or a supported watch page.
- `contextMenus`: Add one top-level item for the thing the user right-clicked: selected text, an image, an X post, a Reddit post, a Xiaohongshu note, a Zhihu answer or article, or a GitHub repository page. The video-note item appears only on a supported YouTube or Bilibili watch page and runs only after the user clicks it.
- `activeTab`: Read the active page only after the user clicks the extension's save action or chooses a save command for selected text, an image, an X post, a Reddit post, a Xiaohongshu note, a Zhihu item, a GitHub repository, a Hacker News post, or a supported watch page.
- `contextMenus`: Add one top-level item for the thing the user right-clicked: selected text, an image, an X post, a Reddit post, a Xiaohongshu note, a Zhihu answer or article, a GitHub repository page, or a Hacker News post detail page. The video-note item appears only on a supported YouTube or Bilibili watch page and runs only after the user clicks it.
- `scripting`: Inject the packaged capture script into the active page after the user initiates a capture.
- `storage`: Store the user's EdgeEver instance URL, API token, and default notebook ID locally.
- Optional host permissions: API calls go only to the EdgeEver instance the user approves. If a page cannot provide an image file, the extension asks for that image's site, or for all sites when the user chooses that option, and uses the access only to download the chosen image. Saving from an X or Xiaohongshu feed may ask for site access to identify the item under the pointer. On Zhihu and Reddit, a content script only remembers the item under the pointer. Its content is read after the user chooses the save command. If that listener is unavailable, the extension asks the user to right-click the same item again. The video-note command does not add a host permission. It runs in the page only after the user clicks it on a supported watch page.
+3
View File
@@ -26,6 +26,9 @@ test("the injected page capture is a standalone classic script", async () => {
const capture = await readFile(join(outDir, "assets/capture.js"), "utf8");
expect(() => new Script(capture)).not.toThrow();
expect(capture).toContain("capturedPage");
const platform = await readFile(join(outDir, "assets/capture-platform.js"), "utf8");
expect(() => new Script(platform)).not.toThrow();
expect(platform).toContain("pagePlatformRead");
} finally {
await rm(outDir, { recursive: true, force: true });
}
+5
View File
@@ -45,6 +45,7 @@ test("registers context menus on install, not on background startup", async () =
"save-reddit",
"save-reddit-link",
"save-video",
"save-hacker-news",
]);
expect(created.find((item) => item.id === "save-xhs")).toMatchObject({
contexts: ["page", "video"],
@@ -81,6 +82,10 @@ test("registers context menus on install, not on background startup", async () =
});
expect(created.find((item) => item.id === "save-selection")?.contexts).toEqual(["selection"]);
expect(created.find((item) => item.id === "save-image")?.contexts).toEqual(["image"]);
for (const id of ["save-hacker-news"]) {
expect(created.find((item) => item.id === id)?.contexts).toEqual(["page"]);
}
expect(created.find((item) => item.id === "save-hacker-news")?.documentUrlPatterns).toEqual(["https://news.ycombinator.com/item?*"]);
const video = created.find((item) => item.id === "save-video");
expect(video).toMatchObject({
contexts: ["page", "video"],
@@ -0,0 +1,3 @@
<!doctype html>
<html><head></head><body><table class="fatitem" border="0"><tbody><tr class="athing submission" id="40380738"><td align="right" valign="top" class="title"><span class="rank"></span></td><td valign="top" class="votelinks"><center><a id="up_40380738" href="vote?id=40380738&amp;how=up&amp;goto=item%3Fid%3D40380738"><div class="votearrow" title="upvote"></div></a></center></td><td class="title"><span class="titleline"><a href="https://hackersearch.net/ask">Show HN: I built a LLM-powered Ask HN: like Perplexity, but for HN comments</a><span class="sitebit comhead"> (<a href="from?site=hackersearch.net"><span class="sitestr">hackersearch.net</span></a>)</span></span></td></tr><tr><td colspan="2"></td><td class="subtext"><span class="subline"><span class="score" id="score_40380738">92 points</span> by <a href="user?id=jnnnthnn" class="hnuser">jnnnthnn</a> <span class="age" title="2024-05-16T17:11:58"><a href="item?id=40380738">on May 16, 2024</a></span> <span id="unv_40380738"></span> | <a href="hide?id=40380738&amp;goto=item%3Fid%3D40380738">hide</a> | <a href="https://hn.algolia.com/?query=Show%20HN%3A%20I%20built%20a%20LLM-powered%20Ask%20HN%3A%20like%20Perplexity%2C%20but%20for%20HN%20comments&amp;type=story&amp;dateRange=all&amp;sort=byDate&amp;storyText=false&amp;prefix&amp;page=0" class="hnpast">past</a> | | <a href="item?id=40380738">39&nbsp;comments</a></span></td></tr><tr><td colspan="2"></td><td><div class="toptext" style="margin-top:4px">Hi HN!<p>I'm Jonathan and I built Ask Hacker Search (<a href="https://hackersearch.net/ask" rel="nofollow">https://hackersearch.net/ask</a>), an LLM-powered version of Hacker News' Ask HN.</p><p>Unlike Ask HN, Ask Hacker Search doesn't solicit new contributions from HN readers. Instead, it leverages Hacker News' historical data to answer questions, and offers LLM-generated summaries of those. I've used it for questions like "Should I use Drizzle or Prisma?" or "What is a good screen capture that allows easy zooming effects on Mac?".</p><p>It is particularly useful when you're interested in understanding HN readers' sentiment about a topic, or when looking for expert insights on topics of interest to HN readers. I've been using it continually while building it, and have found it particularly useful to find software libraries recommended by HN or get quick vibe checks on hot topics.</p><p>This builds on my release of Hacker Search two weeks ago
(<a href="https://news.ycombinator.com/item?id=40238509">https://news.ycombinator.com/item?id=40238509</a>), which offered a semantic search engine over top HN submissions. It's not just a small upgrade: covering comments was the #1 requested feature after that launch, so I rebuilt the near entirety of the product to support that.</p><p>Please try it out and let me know what you think of it! I have to limit the number of LLM summaries each person can get for free, as this is entirely self-funded. If you hit the limit, you can subscribe for more summaries generated by a better model ($8/month), or bring your own compute by running inference on Ollama on your machine!</p></div></td></tr></tbody></table><table class="comment-tree"><tr class="athing comtr" id="40384810"><td class="default"><div class="comment"><div class="commtext">Well done. Matching the HN style with Tailwind is a nice touch.</div></div></td></tr></table></body></html>
@@ -0,0 +1,12 @@
{
"capturedOn": "2026-10-08",
"samples": [
{
"file": "hacker-news.html",
"url": "https://news.ycombinator.com/item?id=40380738",
"state": "Public, signed out; main post with both text and an external link.",
"expected": "Keep title, jnnnthnn, original time, score 92, complete toptext and hackersearch.net/ask. Exclude the separate comment tree."
}
],
"redaction": "Only relevant main-post DOM and one comment excerpt remain. Tests derive synthetic boundary states from this fixture; these variants are not live-site acceptance evidence."
}
+1 -1
View File
@@ -9,7 +9,7 @@
"dev": "vite --config vite.config.ts",
"lint:firefox": "web-ext lint --source-dir dist-firefox --output text",
"package:firefox": "bun run build:firefox && bun run lint:firefox && web-ext build --source-dir dist-firefox --artifacts-dir web-ext-artifacts --overwrite-dest",
"test": "bun test manifest.test.mjs context-menus.test.mjs popup.test.mjs capture-bundle.test.mjs image-clip.test.mjs tweet-clip.test.mjs selection-clip.test.mjs github-repo.test.mjs xhs-clip.test.mjs zhihu-clip.test.mjs reddit-clip.test.mjs video-clip.test.mjs page-images.test.mjs notebook-clip.test.mjs"
"test": "bun test manifest.test.mjs context-menus.test.mjs popup.test.mjs capture-bundle.test.mjs image-clip.test.mjs tweet-clip.test.mjs selection-clip.test.mjs github-repo.test.mjs xhs-clip.test.mjs zhihu-clip.test.mjs reddit-clip.test.mjs video-clip.test.mjs page-images.test.mjs notebook-clip.test.mjs platform-clip.test.mjs platform-save.test.mjs"
},
"dependencies": {
"@mixmark-io/domino": "2.2.0",
+9
View File
@@ -42,6 +42,15 @@ const createClient = ({ failUploadFor = [], failSave = false, session = created,
const onlyFrom = (urls) => async (image) => (urls.includes(image.url) ? pngFile : null);
describe("page clip images", () => {
test("counts every external image while transferring only the first 30", async () => {
const markdown = Array.from({ length: 32 }, (_, index) => `![${index}](https://example.com/${index}.png)`).join("\n\n");
const images = pageImageRefs(markdown, "https://example.com", Infinity);
const { client, calls } = createClient();
const result = await embedPageImages(client, { memoId: "memo", markdown, created, images, download: async () => pngFile });
expect(result).toEqual({ embedded: 30, total: 32 });
expect(calls.uploads).toHaveLength(30);
expect(calls.saves[0].body.contentMarkdown).toContain("![31](https://example.com/31.png)");
});
test("finds remote image addresses, resolving relative ones against the page", () => {
const markdown = [
"![Hero](https://cdn.example.com/hero.png)",
+61
View File
@@ -0,0 +1,61 @@
import { expect, test } from "bun:test";
import { readFileSync } from "node:fs";
import { createDocument } from "@mixmark-io/domino";
import { platformTarget, platformHtmlMarkdown } from "./src/platform-clip.ts";
import { readHackerNews } from "./src/hacker-news-clip.ts";
const labels = { score: "Points at capture", externalLink: "Linked page" };
const hnUrl = "https://news.ycombinator.com/item?id=40380738";
const fixture = (name) => createDocument(readFileSync(new URL(`./fixtures/platforms/${name}.html`, import.meta.url), "utf8"));
test("HN detail URLs retain identity and discard tracking and comment anchors", () => {
expect(platformTarget(`${hnUrl}&utm_source=x#40384810`)).toEqual({ platform: "hacker-news", id: "40380738", url: hnUrl });
for (const url of ["https://news.ycombinator.com/", `${hnUrl}&id=1`, "https://news.ycombinator.com.evil.test/item?id=1", "https://u:p@news.ycombinator.com/item?id=1", "http://news.ycombinator.com/item?id=1", "https://news.ycombinator.com/item?id=0"]) expect(platformTarget(url)).toBeNull();
});
test("HN real DOM keeps text and external link together, metadata and no comments", () => {
const result = readHackerNews(fixture("hacker-news"), hnUrl, labels);
expect(result.ok).toBe(true);
expect(result.clip).toMatchObject({ platform: "hacker-news", id: "40380738", url: hnUrl, author: "jnnnthnn", publishedAt: { value: "2024-05-16T17:11:58", precision: "datetime" }, tags: ["web-clip", "hacker-news"] });
expect(result.clip.title).toStartWith("Show HN: I built a LLM-powered Ask HN");
expect(result.clip.markdown).toContain("Hi HN!");
expect(result.clip.markdown).toContain("Ollama on your machine!");
expect(result.clip.markdown).toContain("Linked page: [https://hackersearch.net/ask](https://hackersearch.net/ask)");
expect(result.clip.markdown).toContain("Points at capture: 92");
expect(result.clip.markdown).not.toContain("Matching the HN style");
});
test("HN allows Ask text and link-only posts and omits missing metadata", () => {
const text = fixture("hacker-news");
text.querySelector(".titleline a").setAttribute("href", "item?id=40380738");
text.querySelector(".score").remove(); text.querySelector(".age").remove();
const ask = readHackerNews(text, hnUrl, labels);
expect(ask.ok).toBe(true);
expect(ask.clip.markdown).not.toContain("Linked page:");
expect(ask.clip.markdown).not.toContain("Points at capture:");
expect(ask.clip.publishedAt).toBeUndefined();
const link = fixture("hacker-news"); link.querySelector(".toptext").remove();
expect(readHackerNews(link, hnUrl, labels).clip.markdown).toContain("Linked page:");
});
test("HN rejects comment, deleted, unreadable and mismatched main posts", () => {
const doc = fixture("hacker-news");
expect(readHackerNews(doc, hnUrl.replace("40380738", "40384810"), labels).ok).toBe(false);
doc.querySelector(".titleline").remove();
expect(readHackerNews(doc, hnUrl, labels).ok).toBe(false);
const deleted = fixture("hacker-news"); deleted.querySelector(".titleline a").textContent = "[deleted]";
expect(readHackerNews(deleted, hnUrl, labels).ok).toBe(false);
});
test("HTML cleanup preserves semantic content and code language on a copy", () => {
const doc = createDocument('<div id="body"><h2>正文</h2><blockquote>引用</blockquote><ul><li>条目</li></ul><pre><code class="language-js">const x = `safe`;\n```</code></pre><a href="/article">链接</a><img data-src="//images.example.com/a.png" onerror="evil()"><script>bad()</script><a href="javascript:evil()">危险链接</a><iframe src="/evil"></iframe><button>分享</button></div>');
const body = doc.querySelector("#body"); const before = body.outerHTML;
const markdown = platformHtmlMarkdown(body, "https://news.ycombinator.com/item?id=40380738");
expect(markdown).toContain("## 正文");
expect(markdown).toContain("> 引用");
expect(markdown).toMatch(/-\s+条目/);
expect(markdown).toContain("````js\nconst x = `safe`;\n```\n````");
expect(markdown).toContain("[链接](https://news.ycombinator.com/article)");
expect(markdown).toContain("https://images.example.com/a.png");
for (const unwanted of ["javascript:", "evil()", "bad()", "分享", "onerror", "<iframe"]) expect(markdown).not.toContain(unwanted);
expect(body.outerHTML).toBe(before);
});
+176
View File
@@ -0,0 +1,176 @@
import { expect, test } from "bun:test";
import { readFileSync } from "node:fs";
import { createContext, runInContext } from "node:vm";
import { fileURLToPath } from "node:url";
import { build } from "esbuild";
import { createWindow } from "@mixmark-io/domino";
const bundle = async (name) => (await build({ entryPoints: [fileURLToPath(new URL(`./src/${name}.ts`, import.meta.url))], bundle: true, platform: "browser", format: "iife", write: false })).outputFiles[0].text;
const [background, captureScript] = await Promise.all([bundle("background"), bundle("capture-platform")]);
const urls = {
"hacker-news": "https://news.ycombinator.com/item?id=40380738",
};
const png = new Uint8Array([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 1, 2, 3]);
// Execute the actual bundled background and injected script. Only browser APIs
// and the instance's HTTP boundary are mocked; extraction and saving are real.
const harness = (options = {}) => {
const platform = options.platform || "hacker-news";
const pageUrl = options.pageUrl || urls[platform];
let currentUrl = pageUrl;
const window = createWindow(readFileSync(new URL(`./fixtures/platforms/${platform}.html`, import.meta.url), "utf8"), pageUrl);
window.getSelection = () => null;
options.prepare?.(window.document, window);
const calls = { creates: [], uploads: [], saves: [], injections: [], feedback: [], requests: [], rejectedMessages: [] };
let onMessage, onClick, menuDone;
const json = (value, status = 200) => Response.json(value, { status });
const fetchMock = async (url, init) => {
calls.requests.push(String(url));
const path = new URL(url).pathname;
if (String(url).startsWith("https://edgeever.test")) {
if (path === "/api/v1/notebooks") {
if (options.navigateDuringNotebookLookup) currentUrl = "https://news.ycombinator.com/item?id=40380739";
return json({ notebooks: [{ id: "nb_inbox", slug: "inbox" }] });
}
if (path === "/api/v1/memos") {
calls.creates.push(JSON.parse(init.body));
return json({ memo: { id: "memo_1", revision: 1, contentHash: "original" } }, 201);
}
if (path.endsWith("/resources")) {
calls.uploads.push(init.body);
return options.failUpload ? json({ error: { message: "upload failed" } }, 500) : json({ resource: { id: `res_${calls.uploads.length}` } });
}
if (path.endsWith("/edit-sessions")) return json({ editSession: { id: "session", baseRevision: options.editConflict ? 2 : 1, baseContentHash: options.editConflict ? "user-edited" : "original" } });
if (path.endsWith("/save")) {
if (options.failRewrite) return json({ error: { message: "conflict" } }, 409);
calls.saves.push(JSON.parse(init.body)); return json({});
}
throw new Error(`Unexpected instance request: ${path}`);
}
return options.failDownload ? new Response("Forbidden", { status: 403 }) : new Response(png, { headers: { "content-type": "image/png" } });
};
let pageContext;
const chrome = {
i18n: { getMessage: (key) => key },
runtime: {
id: "extension-under-test",
onInstalled: { addListener() {} }, onMessage: { addListener(listener) { onMessage = listener; } },
openOptionsPage: async () => {},
sendMessage: async (message) => {
if (options.spoofMessages && message.type === "pagePlatformRead") {
for (const overrides of [{ tab: { id: 999 } }, { frameId: 3 }, { id: "other-extension" }]) {
const fake = { ...message, result: { ok: true, clip: { ...message.result.clip, title: "Wrong sender" } } };
onMessage(fake, { id: chrome.runtime.id, tab: { id: 7 }, frameId: 0, url: currentUrl, ...overrides }, () => {});
calls.rejectedMessages.push(overrides);
}
onMessage({ ...message, requestId: "unrelated", result: { ok: false } }, { id: chrome.runtime.id, tab: { id: 7 }, frameId: 0, url: currentUrl }, () => {});
}
onMessage(message, { id: chrome.runtime.id, tab: { id: 7 }, frameId: 0, url: currentUrl }, () => {});
},
},
storage: { local: { get: async () => ({ instanceUrl: "https://edgeever.test", token: "test-token", notebookId: "" }) } },
permissions: { contains: async () => !options.denyInstance },
tabs: { query: async () => [{ id: 7, url: currentUrl }], get: async () => ({ id: 7, url: currentUrl }) },
windows: { onRemoved: { addListener() {} } },
contextMenus: { onClicked: { addListener(listener) { onClick = listener; } } },
scripting: { executeScript: async ({ func, args = [], files, target }) => {
calls.injections.push({ files, target });
if (options.denyScripting) throw new Error("Cannot access contents of the page. Missing host permission.");
if (func) {
if (args[0]?.action === "toast") { calls.feedback.push(args[0]); if (calls.feedback.length > 1) menuDone?.(args[0]); return []; }
pageContext.__args = args;
return [{ result: await runInContext(`(${func.toString()})(...__args)`, pageContext) }];
}
if (files?.[0] === "assets/capture-platform.js") runInContext(captureScript, pageContext);
else if (files?.[0] !== "assets/capture-image.js") throw new Error(`Unexpected generic fallback: ${files}`);
return [];
} },
action: { setBadgeBackgroundColor: async () => {}, setBadgeText: async () => {} },
};
pageContext = createContext({ window, document: window.document, location: { get href() { return currentUrl; } }, URL, chrome, setTimeout, clearTimeout, fetch: fetchMock, AbortSignal, Blob, Uint8Array });
runInContext(background, createContext({ chrome, fetch: fetchMock, URL, crypto, setTimeout, clearTimeout, setInterval, clearInterval, AbortSignal, AbortController, Blob, File, FormData, Uint8Array, ArrayBuffer, Response }));
return {
calls,
toolbar: () => new Promise((resolve) => onMessage({ type: "captureCurrentPage" }, { id: chrome.runtime.id }, resolve)),
menu: () => new Promise((resolve) => { menuDone = resolve; onClick({ menuItemId: `save-${platform}`, pageUrl, frameId: 0 }, { id: 7, url: pageUrl }); }),
};
};
for (const platform of Object.keys(urls)) {
test(`${platform}: toolbar and dedicated menu save the same note through the real parser`, async () => {
const toolbar = harness({ platform }); const menu = harness({ platform });
expect((await toolbar.toolbar()).ok).toBe(true);
await menu.menu();
expect(toolbar.calls.creates).toHaveLength(1);
expect(menu.calls.creates).toHaveLength(1);
const normalize = (note) => ({ ...note, contentMarkdown: note.contentMarkdown.replace(/capturedAtLabel: [^\n]+/, "capturedAtLabel: <capture time>") });
expect(normalize(menu.calls.creates[0])).toEqual(normalize(toolbar.calls.creates[0]));
expect(toolbar.calls.creates[0].tags).toEqual(["web-clip", platform]);
expect(toolbar.calls.creates[0].notebookId).toBe("nb_inbox");
expect(toolbar.calls.requests.some((url) => /\.(mp3|m4a|mp4)(\?|$)/.test(url))).toBe(false);
});
}
test("a new platform toolbar still saves a valid selection instead of its full article", async () => {
const h = harness({ platform: "hacker-news", prepare: (doc, window) => {
window.getSelection = () => ({ rangeCount: 1, toString: () => "Only this passage", getRangeAt: () => ({ cloneContents: () => {
const fragment = doc.createDocumentFragment(); const p = doc.createElement("p"); p.textContent = "Only this passage"; fragment.appendChild(p); return fragment;
} }) });
} });
expect((await h.toolbar()).ok).toBe(true);
expect(h.calls.creates[0].tags).toEqual(["web-clip"]);
expect(h.calls.creates[0].contentMarkdown).toContain("Only this passage");
expect(h.calls.creates[0].contentMarkdown).not.toContain("Ollama on your machine!");
});
test("request, extension, tab and frame correlation reject unrelated responses", async () => {
const h = harness({ spoofMessages: true });
expect((await h.toolbar()).ok).toBe(true);
expect(h.calls.rejectedMessages).toHaveLength(3);
expect(h.calls.creates).toHaveLength(1);
expect(h.calls.creates[0].title).toStartWith("Show HN");
});
for (const failure of ["failDownload", "failUpload", "failRewrite", "editConflict"]) {
test(`${failure}: keeps the saved body, reports external images, and never creates twice`, async () => {
const h = harness({ [failure]: true, prepare: (doc) => { doc.querySelector(".toptext").innerHTML += '<img src="https://images.test/body.png">'; } });
expect(await h.toolbar()).toEqual({ ok: true, message: "platformImagesPartial" });
expect(h.calls.creates).toHaveLength(1);
expect(h.calls.saves).toHaveLength(0);
expect(h.calls.creates[0].contentMarkdown).toContain("images.test/body.png");
});
}
test("images beyond the limit count towards partial status after 30 successful uploads", async () => {
const h = harness({ platform: "hacker-news", prepare: (doc) => {
const body = doc.querySelector(".toptext");
body.innerHTML += Array.from({ length: 32 }, (_, i) => `<img src="https://images.test/${i}.png">`).join("");
} });
expect(await h.toolbar()).toEqual({ ok: true, message: "platformImagesPartial" });
expect(h.calls.uploads).toHaveLength(30);
expect(h.calls.creates).toHaveLength(1);
expect(h.calls.saves[0].contentMarkdown).toContain("/api/v1/resources/res_1/blob");
expect(h.calls.saves[0].contentMarkdown).toContain("https://images.test/31.png");
});
for (const platform of Object.keys(urls)) {
test(`${platform}: unreadable content never falls back to a whole-page note`, async () => {
const h = harness({ platform, prepare: (doc) => { doc.body.innerHTML = "<h1>Sign in</h1><p>Comments and recommendations</p>"; } });
expect(await h.toolbar()).toEqual({ ok: false, message: "platformUnreadable" });
expect(h.calls.creates).toHaveLength(0);
});
}
test("navigation while resolving the notebook cancels the old content before creation", async () => {
const h = harness({ platform: "hacker-news", navigateDuringNotebookLookup: true });
expect(await h.toolbar()).toEqual({ ok: false, message: "platformPageChanged" });
expect(h.calls.creates).toHaveLength(0);
});
for (const [option, message] of [["denyInstance", "instancePermissionRequired"], ["denyScripting", "pageAccessDenied"]]) {
test(`${option}: gives an actionable permission error without creating a note`, async () => {
const h = harness({ [option]: true });
expect(await h.toolbar()).toEqual({ ok: false, message });
expect(h.calls.creates).toHaveLength(0);
});
}
+1
View File
@@ -21,6 +21,7 @@
<span class="platform-tag" data-i18n="platformXiaohongshu">小红书</span>
<span class="platform-tag" data-i18n="platformZhihu">知乎</span>
<span class="platform-tag">GitHub</span>
<span class="platform-tag">Hacker News</span>
</div>
</div>
<button id="save" class="save-button" type="button" disabled>
@@ -3,7 +3,7 @@
"message": "EdgeEver Web Clipper"
},
"extensionDescription": {
"message": "Save the current webpage, selected content, or a right-clicked image to your self-hosted EdgeEver instance."
"message": "Save webpages, selections, images, and Hacker News posts to your self-hosted EdgeEver instance."
},
"actionTitle": {
"message": "Save to EdgeEver"
@@ -560,5 +560,41 @@
},
"videoNoteFallbackTitle": {
"message": "Video note"
},
"saveHackerNewsToEdgeEver": {
"message": "Save Hacker News post to EdgeEver"
},
"savingPlatformClip": {
"message": "Saving this page to EdgeEver…"
},
"platformNotDetail": {
"message": "Open a Hacker News post detail page, then try again."
},
"platformUnreadable": {
"message": "The content is unavailable. Sign in if needed, wait for the detail page to load, then try again."
},
"platformPageChanged": {
"message": "The page changed during capture. Nothing was saved; try again on the intended detail page."
},
"platformImagesPartial": {
"message": "Text saved. Some images still use external links."
},
"platformSavedWithImages": {
"message": "Saved to EdgeEver with all images archived."
},
"platformAuthorLabel": {
"message": "Author"
},
"platformPublishedAtLabel": {
"message": "Published"
},
"platformRelativeTimeLabel": {
"message": "as displayed; relative time"
},
"platformScoreLabel": {
"message": "Points at capture"
},
"platformExternalLinkLabel": {
"message": "Linked page"
}
}
@@ -3,7 +3,7 @@
"message": "EdgeEver Web Clipper"
},
"extensionDescription": {
"message": "表示中のページ、選択した内容、または右クリックした画像を、自分で運用する EdgeEver に保存します。"
"message": "ウェブページ、選択した文章、画像、Hacker News の投稿をセルフホストの EdgeEver に保存します。"
},
"actionTitle": {
"message": "EdgeEver に保存"
@@ -560,5 +560,41 @@
},
"videoNoteFallbackTitle": {
"message": "動画ノート"
},
"saveHackerNewsToEdgeEver": {
"message": "Hacker News の投稿を EdgeEver に保存"
},
"savingPlatformClip": {
"message": "このページを EdgeEver に保存中…"
},
"platformNotDetail": {
"message": "Hacker News の投稿詳細ページを開いて、もう一度お試しください。"
},
"platformUnreadable": {
"message": "内容を読み取れません。必要に応じてログインし、詳細ページの読み込み後にもう一度お試しください。"
},
"platformPageChanged": {
"message": "保存中にページが切り替わりました。保存は行われていません。目的の詳細ページで再試行してください。"
},
"platformImagesPartial": {
"message": "本文を保存しました。一部の画像は外部リンクのままです。"
},
"platformSavedWithImages": {
"message": "すべての画像とともに EdgeEver に保存しました。"
},
"platformAuthorLabel": {
"message": "投稿者"
},
"platformPublishedAtLabel": {
"message": "公開日時"
},
"platformRelativeTimeLabel": {
"message": "ページの表示どおりの相対時刻"
},
"platformScoreLabel": {
"message": "保存時のポイント"
},
"platformExternalLinkLabel": {
"message": "元の外部リンク"
}
}
@@ -3,7 +3,7 @@
"message": "EdgeEver Web Clipper"
},
"extensionDescription": {
"message": "Zapisuj bieżącą stronę, zaznaczony fragment lub obraz z menu kontekstowego w swojej samodzielnie hostowanej instancji EdgeEver."
"message": "Zapisuj strony, wybrany tekst, obrazy i wpisy Hacker News w swojej instancji EdgeEver."
},
"actionTitle": {
"message": "Zapisz w EdgeEver"
@@ -560,5 +560,41 @@
},
"videoNoteFallbackTitle": {
"message": "Notatka z wideo"
},
"saveHackerNewsToEdgeEver": {
"message": "Zapisz wpis Hacker News w EdgeEver"
},
"savingPlatformClip": {
"message": "Zapisywanie strony w EdgeEver…"
},
"platformNotDetail": {
"message": "Otwórz stronę szczegółów wpisu Hacker News i spróbuj ponownie."
},
"platformUnreadable": {
"message": "Nie można odczytać treści. W razie potrzeby zaloguj się, poczekaj na załadowanie strony i spróbuj ponownie."
},
"platformPageChanged": {
"message": "Strona zmieniła się podczas zapisywania. Nic nie zapisano. Spróbuj ponownie na właściwej stronie."
},
"platformImagesPartial": {
"message": "Tekst zapisano. Część obrazów nadal korzysta z linków zewnętrznych."
},
"platformSavedWithImages": {
"message": "Zapisano w EdgeEver wraz ze wszystkimi obrazami."
},
"platformAuthorLabel": {
"message": "Autor"
},
"platformPublishedAtLabel": {
"message": "Opublikowano"
},
"platformRelativeTimeLabel": {
"message": "czas względny wyświetlony na stronie"
},
"platformScoreLabel": {
"message": "Punkty w chwili zapisu"
},
"platformExternalLinkLabel": {
"message": "Link źródłowy"
}
}
@@ -3,7 +3,7 @@
"message": "EdgeEver 网页剪藏"
},
"extensionDescription": {
"message": "将当前网页、选中内容,或右键图片保存到你自托管的 EdgeEver 实例。"
"message": "将网页、选中文字、图片及 Hacker News 主帖保存到你自托管的 EdgeEver 实例。"
},
"actionTitle": {
"message": "保存到 EdgeEver"
@@ -560,5 +560,41 @@
},
"videoNoteFallbackTitle": {
"message": "视频笔记"
},
"saveHackerNewsToEdgeEver": {
"message": "保存 Hacker News 主帖到 EdgeEver"
},
"savingPlatformClip": {
"message": "正在保存当前内容到 EdgeEver…"
},
"platformNotDetail": {
"message": "请打开 Hacker News 主帖详情页后重试。"
},
"platformUnreadable": {
"message": "无法读取目标内容。请按需登录,等待详情页加载完成后重试。"
},
"platformPageChanged": {
"message": "剪藏时页面内容已切换,尚未保存。请在目标详情页重试。"
},
"platformImagesPartial": {
"message": "正文已保存,部分图片仍为外链。"
},
"platformSavedWithImages": {
"message": "已保存到 EdgeEver,图片已全部归档。"
},
"platformAuthorLabel": {
"message": "作者"
},
"platformPublishedAtLabel": {
"message": "发布时间"
},
"platformRelativeTimeLabel": {
"message": "页面原文,相对时间"
},
"platformScoreLabel": {
"message": "采集时积分"
},
"platformExternalLinkLabel": {
"message": "原始外链"
}
}
+145 -4
View File
@@ -89,6 +89,15 @@ import {
import type { OutlineAttempt, VideoNoteToast } from "./video/types";
import { youtubeCaptureFromRead, youtubeTargetFromUrl } from "./video/youtube";
import { t } from "./i18n";
import {
isPlatformClip,
markdownText,
markdownUrl,
PLATFORM_MENUS,
platformTarget,
publishedTimeText,
type PlatformTarget,
} from "./platform-clip";
type CapturedPage = {
title: string;
@@ -451,11 +460,13 @@ const embedClipImages = async (
tabId: number | null,
frameId: number | null,
) => {
if (!memo?.id || typeof memo.revision !== "number" || typeof memo.contentHash !== "string") return;
const images = pageImageRefs(markdown, pageUrl);
if (images.length === 0) return;
// Count all remote images, including those beyond the transfer cap. The new
// platform path uses this result to report partial archival honestly.
const images = pageImageRefs(markdown, pageUrl, Number.POSITIVE_INFINITY);
const unchanged = { embedded: 0, total: images.length };
if (!memo?.id || typeof memo.revision !== "number" || typeof memo.contentHash !== "string" || images.length === 0) return unchanged;
try {
await embedPageImages(imageNoteClient(settings), {
return await embedPageImages(imageNoteClient(settings), {
memoId: memo.id,
markdown,
created: { revision: memo.revision, contentHash: memo.contentHash },
@@ -465,6 +476,7 @@ const embedClipImages = async (
} catch {
// The note is already saved; reporting a failure here would invite a
// duplicate save. Its images keep their remote addresses.
return unchanged;
}
};
@@ -544,6 +556,11 @@ let pendingCapture: ((page: CapturedPage) => void) | null = null;
const pendingImageReads = new Map<string, (result: unknown) => void>();
const pendingTweetReads = new Map<string, (result: unknown) => void>();
const pendingGithubReads = new Map<string, (result: unknown) => void>();
const pendingPlatformReads = new Map<string, {
tabId: number;
target: PlatformTarget;
resolve: (result: unknown) => void;
}>();
const pendingXhsReads = new Map<string, (result: unknown) => void>();
const pendingZhihuReads = new Map<string, (result: unknown) => void>();
const pendingRedditReads = new Map<string, (result: unknown) => void>();
@@ -1263,6 +1280,102 @@ const saveGithubRepoFromMenu = async (
}
};
const platformReadErrors = {
"not-target": "platformNotDetail",
unreadable: "platformUnreadable",
changed: "platformPageChanged",
} as const;
const describePlatformError = (error: unknown) => {
const message = error instanceof Error ? error.message : "";
if (Object.values(platformReadErrors).some((key) => message === t(key))) return message;
return describeGithubError(error);
};
const readPlatformFromPage = async (tabId: number, target: PlatformTarget, selectionFirst: boolean) => {
const requestId = crypto.randomUUID();
return new Promise<unknown>((resolve, reject) => {
const timeout = setTimeout(() => {
pendingPlatformReads.delete(requestId);
reject(new Error(t("captureTimeout")));
}, 10_000);
pendingPlatformReads.set(requestId, { tabId, target, resolve: (result) => {
clearTimeout(timeout);
pendingPlatformReads.delete(requestId);
resolve(result);
} });
const inject = async () => {
await chrome.scripting.executeScript({
target: { tabId, frameIds: [0] },
func: (payload: unknown) => {
(globalThis as { __edgeeverPlatformClipPayload?: unknown }).__edgeeverPlatformClipPayload = payload;
},
args: [{ requestId, url: target.url, selectionFirst }],
});
await chrome.scripting.executeScript({ target: { tabId, frameIds: [0] }, files: ["assets/capture-platform.js"] });
};
void inject().catch((error: unknown) => {
clearTimeout(timeout);
pendingPlatformReads.delete(requestId);
reject(new Error(describeCaptureError(error)));
});
});
};
const performPlatformSave = async (tabId: number, pageUrl: string, selectionFirst: boolean) => {
const target = platformTarget(pageUrl);
if (!target) throw new Error(t("platformNotDetail"));
const settings = await ensureClipperReady();
const result = await readPlatformFromPage(tabId, target, selectionFirst);
if (!result || typeof result !== "object") throw new Error(t("platformUnreadable"));
const record = result as { ok?: unknown; reason?: unknown; clip?: unknown; selection?: unknown };
if (record.ok !== true) {
const key = typeof record.reason === "string" && Object.hasOwn(platformReadErrors, record.reason)
? platformReadErrors[record.reason as keyof typeof platformReadErrors] : "platformUnreadable";
throw new Error(t(key));
}
const clip = isPlatformClip(record.clip, target) ? record.clip : null;
const selection = selectionFirst && isCapturedPage(record.selection) && record.selection.kind === "selection"
&& record.selection.url === target.url && record.selection.markdown.trim() ? record.selection : null;
if (!clip && !selection) throw new Error(t("platformUnreadable"));
const capturedAt = new Date().toISOString();
const contentMarkdown = clip ? [
`# ${markdownText(clip.title)}`,
clip.author ? `${t("platformAuthorLabel")}: ${markdownText(clip.author)}` : "",
clip.publishedAt ? `${t("platformPublishedAtLabel")}: ${markdownText(publishedTimeText(clip.publishedAt, t("platformRelativeTimeLabel")))}` : "",
`${t("sourceLabel")}: [${markdownText(clip.url)}](${markdownUrl(clip.url)})`,
`${t("capturedAtLabel")}: ${capturedAt}`,
"---", clip.markdown,
].filter(Boolean).join("\n\n") : toMarkdown(selection!);
const notebookId = await notebookForClip(settings);
// Notebook lookup and capture can both outlive a navigation. Bind the save
// to the detail page the user clicked.
const current = await chrome.tabs.get(tabId);
if (platformTarget(current.url || "")?.url !== target.url) throw new Error(t("platformPageChanged"));
const created = await edgeEverRequest<{ memo?: CreatedClipMemo }>(settings, "/api/v1/memos", {
method: "POST",
body: JSON.stringify({ notebookId, title: (clip?.title || selection!.title).slice(0, 120), contentMarkdown, tags: clip?.tags || ["web-clip"] }),
});
const images = await embedClipImages(settings, created.memo, contentMarkdown, target.url, tabId, 0);
const message = images.embedded < images.total ? t("platformImagesPartial")
: images.total > 0 ? t("platformSavedWithImages") : t("savedToEdgeEver");
return { ok: true, message };
};
const savePlatformFromMenu = async (info: { pageUrl?: string; frameId?: number }, tab?: { id?: number; url?: string }) => {
const tabId = typeof tab?.id === "number" ? tab.id : null;
try {
if (tabId === null || (info.frameId ?? 0) !== 0) throw new Error(t("platformNotDetail"));
await showFeedback(tabId, 0, t("savingPlatformClip"), "success");
const result = await performPlatformSave(tabId, info.pageUrl || tab?.url || "", false);
await showFeedback(tabId, 0, result.message, "success");
} catch (error) {
const message = describePlatformError(error);
if (message === t("completePluginConfiguration") || message === t("instancePermissionRequired")) await chrome.runtime.openOptionsPage();
await showFeedback(tabId, 0, message, "error");
}
};
const isCapturedPage = (value: unknown): value is CapturedPage => {
if (!value || typeof value !== "object") return false;
const record = value as Partial<CapturedPage>;
@@ -1593,11 +1706,23 @@ const createClipMenus = () => {
}, () => {
void chrome.runtime.lastError;
});
for (const menu of PLATFORM_MENUS) {
chrome.contextMenus.create({
id: menu.id,
title: t(menu.title),
contexts: ["page"],
documentUrlPatterns: [...menu.patterns],
}, () => { void chrome.runtime.lastError; });
}
};
chrome.runtime.onInstalled.addListener(registerClipMenus);
chrome.contextMenus.onClicked.addListener((info: { menuItemId?: string | number; srcUrl?: string; pageUrl?: string; linkUrl?: string; frameId?: number; selectionText?: string }, tab?: { id?: number; url?: string; title?: string }) => {
if (PLATFORM_MENUS.some((menu) => menu.id === info.menuItemId)) {
void enqueueClip(() => savePlatformFromMenu(info, tab));
return;
}
if (info.menuItemId === SELECTION_MENU_ID) {
void enqueueClip(() => saveSelectionFromMenu(info, tab));
return;
@@ -1649,6 +1774,14 @@ chrome.windows.onRemoved.addListener((windowId: number) => {
});
chrome.runtime.onMessage.addListener((message: { type?: string; page?: CapturedPage; requestId?: string; result?: unknown }, _sender: unknown, sendResponse: (response: unknown) => void) => {
if (message.type === "pagePlatformRead" && message.requestId) {
const pending = pendingPlatformReads.get(message.requestId);
const sender = _sender as { id?: string; tab?: { id?: number }; frameId?: number; url?: string } | undefined;
if (pending && sender && sender.id === chrome.runtime.id && sender.tab?.id === pending.tabId && sender.frameId === 0) {
pending.resolve(platformTarget(sender.url || "")?.url === pending.target.url ? message.result : { ok: false, reason: "changed" });
}
return false;
}
if (message.type === "capturedPage" && message.page) {
pendingCapture?.(message.page);
pendingCapture = null;
@@ -1801,6 +1934,14 @@ chrome.runtime.onMessage.addListener((message: { type?: string; page?: CapturedP
}
const pageUrl = tab.url || "";
if (platformTarget(pageUrl)) {
try {
sendResponse(await enqueueClip(() => performPlatformSave(tab.id, pageUrl, true)));
} catch (error) {
sendResponse({ ok: false, message: describePlatformError(error) });
}
return;
}
if (youtubeTargetFromUrl(pageUrl) || bilibiliTargetFromUrl(pageUrl)) {
const result = await enqueueClip(() => performVideoSave(settings, tab.id, null, pageUrl));
sendResponse(result.created ? { ok: true, message: result.message } : { ok: false, message: result.message });
+42
View File
@@ -0,0 +1,42 @@
import { readHackerNews } from "./hacker-news-clip";
import { markdownText, platformHtmlMarkdown, platformTarget, type PlatformLabels, type PlatformRead } from "./platform-clip";
// Built as a standalone IIFE and injected only after a user action.
(() => {
const root = globalThis as typeof globalThis & {
__edgeeverPlatformClipPayload?: { requestId?: string; url?: string; selectionFirst?: boolean };
};
const payload = root.__edgeeverPlatformClipPayload;
delete root.__edgeeverPlatformClipPayload;
if (!payload?.requestId || !payload.url) return;
const requestId = payload.requestId;
const expected = platformTarget(payload.url);
if (!expected) return;
const t = (key: string) => chrome.i18n.getMessage(key) || key;
const labels: PlatformLabels = {
score: t("platformScoreLabel"), externalLink: t("platformExternalLinkLabel"),
};
const samePage = () => platformTarget(location.href)?.url === expected.url;
const read = (): PlatformRead => {
if (!samePage()) return { ok: false, reason: "changed" };
return readHackerNews(document, location.href, labels);
};
const finish = (result: unknown) => {
void chrome.runtime.sendMessage({ type: "pagePlatformRead", requestId, result });
};
const capture = async () => {
if (!samePage()) return { ok: false, reason: "changed" };
const selection = payload.selectionFirst ? window.getSelection() : null;
if (selection?.rangeCount && selection.toString().trim()) {
const selected = document.createElement("div");
for (let index = 0; index < selection.rangeCount; index += 1) selected.appendChild(selection.getRangeAt(index).cloneContents());
const markdown = platformHtmlMarkdown(selected, expected.url)
|| selection.toString().trim().split(/\r?\n/).map(markdownText).join("\n");
return { ok: true, selection: { title: document.title.trim() || expected.platform, url: expected.url, markdown, kind: "selection" } };
}
return read();
};
void capture().then((result) => finish(samePage() ? result : { ok: false, reason: "changed" }))
.catch(() => finish({ ok: false, reason: "unreadable" }));
})();
+34
View File
@@ -0,0 +1,34 @@
import { markdownText, markdownUrl, platformHtmlMarkdown, platformTarget, platformText, publishedTime, safePlatformUrl, type PlatformLabels, type PlatformRead } from "./platform-clip";
export const readHackerNews = (doc: Document, pageUrl: string, labels: PlatformLabels): PlatformRead => {
const target = platformTarget(pageUrl);
if (target?.platform !== "hacker-news") return { ok: false, reason: "not-target" };
const root = doc.querySelector(".fatitem");
const story = root?.querySelector("tr.athing");
const titleLink = story?.querySelector(".titleline > a");
const title = platformText(titleLink);
// A comment's item page also has .fatitem and .athing, but no titleline.
if (!story || !titleLink || !title || /^(\[deleted\]|\[dead\])$/i.test(title)) return { ok: false, reason: "unreadable" };
if (story.id !== target.id) return { ok: false, reason: "changed" };
const metadata = story.nextElementSibling?.querySelector(".subtext");
const body = root?.querySelector(".toptext");
const markdown = body ? platformHtmlMarkdown(body, target.url) : "";
const linked = safePlatformUrl(titleLink.getAttribute("href") || "", target.url);
const externalLink = linked && platformTarget(linked)?.url !== target.url ? linked : "";
if (!markdown && !externalLink) return { ok: false, reason: "unreadable" };
const score = platformText(metadata?.querySelector(`#score_${target.id}`));
const points = /^(\d+)\s+points?$/.exec(score)?.[1];
return {
ok: true,
clip: {
...target, title, author: platformText(metadata?.querySelector(".hnuser")) || undefined,
publishedAt: publishedTime(metadata?.querySelector(".age") ?? null),
markdown: [
points === undefined ? "" : `${markdownText(labels.score)}: ${points}`,
externalLink ? `${markdownText(labels.externalLink)}: [${markdownText(externalLink)}](${markdownUrl(externalLink)})` : "",
markdown,
].filter(Boolean).join("\n\n"),
tags: ["web-clip", target.platform], completeness: "complete",
},
};
};
+1 -1
View File
@@ -204,7 +204,7 @@ export const embedPageImages = async (
const uploaded: Array<{ href: string; resourceId: string }> = [];
let remainingBytes = MAX_PAGE_IMAGE_TOTAL_BYTES;
for (const image of input.images) {
for (const image of input.images.slice(0, MAX_PAGE_IMAGES)) {
if (transfers.aborted || remainingBytes <= 0) break;
try {
const maxBytes = Math.min(MAX_IMAGE_BYTES, remainingBytes);
+116
View File
@@ -0,0 +1,116 @@
import TurndownService from "turndown";
export type ClipPlatform = "hacker-news";
export type PlatformTarget = { platform: ClipPlatform; id: string; url: string };
export type PublishedTime = { value: string; precision: "datetime" | "date" | "relative" };
export type PlatformClip = PlatformTarget & {
title: string;
markdown: string;
author?: string;
publishedAt?: PublishedTime;
tags: string[];
completeness: "complete";
};
export type PlatformRead = { ok: true; clip: PlatformClip } | {
ok: false;
reason: "not-target" | "unreadable" | "changed";
};
export type PlatformLabels = {
score: string;
externalLink: string;
};
export const PLATFORM_MENUS = [
{ id: "save-hacker-news", title: "saveHackerNewsToEdgeEver", patterns: ["https://news.ycombinator.com/item?*"] },
] as const;
export const platformTarget = (pageUrl: string): PlatformTarget | null => {
let url: URL;
try { url = new URL(pageUrl); } catch { return null; }
if (url.protocol !== "https:" || url.username || url.password || url.port) return null;
if (url.hostname === "news.ycombinator.com" && url.pathname === "/item") {
const ids = url.searchParams.getAll("id");
if (ids.length !== 1 || !/^[1-9]\d*$/.test(ids[0])) return null;
return { platform: "hacker-news", id: ids[0], url: `https://news.ycombinator.com/item?id=${ids[0]}` };
}
return null;
};
export const platformText = (node: Node | null | undefined) => (node?.textContent ?? "").replace(/\s+/g, " ").trim();
export const markdownText = (value: string) => value.replace(/[\r\n]+/g, " ").replace(/([\\`*_{}[\]<>])/g, "\\$1");
export const markdownUrl = (value: string) => value.replace(/[\\ ()<>]/g, (char) => encodeURIComponent(char).replace("(", "%28").replace(")", "%29"));
export const safePlatformUrl = (value: string, base: string, mailto = false) => {
if (!value.trim()) return "";
try {
const url = new URL(value, base);
return (url.protocol === "http:" || url.protocol === "https:" || (mailto && url.protocol === "mailto:"))
&& !url.username && !url.password ? url.href : "";
} catch { return ""; }
};
/** Keep the precision of the page's time; never substitute the capture time. */
export const publishedTime = (node: Element | null): PublishedTime | undefined => {
if (!node) return undefined;
const machine = (node.getAttribute("datetime") || node.getAttribute("title") || "").trim();
if (/^\d{4}-\d{2}-\d{2}$/.test(machine)) return { value: machine, precision: "date" };
// HN's ISO-shaped title has no zone; preserve it instead of guessing one.
const normalized = machine.replace(/^(\d{4}-\d{2}-\d{2}) /, "$1T").replace(/ (?=[+-]\d{2}:\d{2}$)/, "");
if (/^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d+)?(?:Z|[+-]\d{2}:\d{2})?$/.test(normalized)) {
return { value: normalized, precision: "datetime" };
}
const value = platformText(node);
return value ? { value, precision: "relative" } : undefined;
};
export const publishedTimeText = (time: PublishedTime, relativeLabel: string) =>
time.precision === "relative" ? `${time.value} (${relativeLabel})` : time.value;
/** Converts a detached copy only. Links and images never keep active protocols. */
export const platformHtmlMarkdown = (body: Element | DocumentFragment, baseUrl: string): string => {
const copy = body.cloneNode(true) as Element | DocumentFragment;
Array.from(copy.querySelectorAll("script, style, noscript, template, iframe, object, embed, canvas, svg, video, audio, source, link, meta, form, input, button, textarea, select, [hidden], [aria-hidden='true']"))
.forEach((node) => node.remove());
for (const node of Array.from(copy.querySelectorAll("*"))) {
for (const attribute of Array.from(node.attributes)) {
if (/^on/i.test(attribute.name) || attribute.name === "style") node.removeAttribute(attribute.name);
}
}
for (const link of Array.from(copy.querySelectorAll("a"))) {
const url = safePlatformUrl(link.getAttribute("href") || "", baseUrl, true);
if (url) link.setAttribute("href", markdownUrl(url));
else link.removeAttribute("href");
}
for (const img of Array.from(copy.querySelectorAll("img"))) {
const url = ["data-original", "data-src", "src"].map((name) => safePlatformUrl(img.getAttribute(name) || "", baseUrl)).find(Boolean);
if (url) img.setAttribute("src", markdownUrl(url));
else img.remove();
img.removeAttribute("srcset");
}
const turndown = new TurndownService({ bulletListMarker: "-", codeBlockStyle: "fenced", headingStyle: "atx", linkStyle: "inlined" });
turndown.addRule("platformCode", {
filter: "pre",
replacement: (_content, node) => {
const element = node as HTMLElement;
const code = element.querySelector("code");
const language = /(?:^|\s)(?:language|lang)-([a-zA-Z0-9_+-]+)(?=\s|$)/.exec(`${code?.className ?? ""} ${element.className}`)?.[1] || "";
const text = (code ?? element).textContent?.replace(/\n$/, "") || "";
const fence = "`".repeat(Math.max(3, ...[...text.matchAll(/`+/g)].map((match) => match[0].length + 1)));
return `\n\n${fence}${language}\n${text}\n${fence}\n\n`;
},
});
return turndown.turndown(copy as HTMLElement).trim();
};
export const isPlatformClip = (value: unknown, expected: PlatformTarget): value is PlatformClip => {
if (!value || typeof value !== "object") return false;
const clip = value as Partial<PlatformClip>;
const time = clip.publishedAt;
return clip.platform === expected.platform && clip.id === expected.id && clip.url === expected.url
&& typeof clip.title === "string" && !!clip.title.trim() && clip.title.length <= 10_000
&& typeof clip.markdown === "string" && !!clip.markdown.trim() && clip.markdown.length <= 2_000_000
&& (clip.author === undefined || typeof clip.author === "string")
&& (!time || (typeof time.value === "string" && ["datetime", "date", "relative"].includes(time.precision)))
&& Array.isArray(clip.tags) && clip.tags.length === 2 && clip.tags[0] === "web-clip" && clip.tags[1] === expected.platform
&& clip.completeness === "complete";
};
+17 -15
View File
@@ -22,21 +22,23 @@ export default defineConfig({
// chrome.scripting.executeScript({ files }) loads classic scripts, so
// capture must not inherit Vite's shared ESM chunks.
const capture = await buildStandaloneScript({
entryPoints: [fileURLToPath(new URL("./src/capture.ts", import.meta.url))],
outfile: "capture.js",
bundle: true,
format: "iife",
platform: "browser",
target: "es2022",
minify: true,
write: false,
});
this.emitFile({
type: "asset",
fileName: "assets/capture.js",
source: capture.outputFiles[0].contents,
});
for (const name of ["capture", "capture-platform"]) {
const capture = await buildStandaloneScript({
entryPoints: [fileURLToPath(new URL(`./src/${name}.ts`, import.meta.url))],
outfile: `${name}.js`,
bundle: true,
format: "iife",
platform: "browser",
target: "es2022",
minify: true,
write: false,
});
this.emitFile({
type: "asset",
fileName: `assets/${name}.js`,
source: capture.outputFiles[0].contents,
});
}
},
},
],
+15
View File
@@ -0,0 +1,15 @@
# Hacker News clipping
Open a main-post detail page at `https://news.ycombinator.com/item?id=<id>`. Use **Clip current page**, or right-click the page and choose **Save Hacker News post to EdgeEver**. A selected passage takes priority for the toolbar action; the dedicated page menu saves the main post.
The note keeps the title, author, original publication time as supplied by the page, points at capture, external link and main-post text when available. Ask HN, Show HN and link-only posts are supported. Comments are excluded. Each full-post note has `web-clip` and `hacker-news` tags.
The linked article is recorded as a link; its content is not fetched. Missing metadata is omitted. Comment detail pages, deleted posts and unreadable content are rejected. A page change during capture cancels saving. No additional persistent site permission is added.
Images in the saved body use the existing attachment workflow. If copying fails or reaches its limit, text remains saved and the message explains that some images still use external links. A concurrent edit is preserved.
## Verification
Automated coverage uses a redacted public HN DOM fixture, parser and bundled background/injection tests, and a Chromium extension test against the local API that reopens the saved note. The image-storage case adds a synthetic image to the HN fixture. These checks do not replace live-site acceptance across Chrome, Edge and Firefox or store publication.
[简体中文](browser-clipper-hacker-news.zh-CN.md)
+15
View File
@@ -0,0 +1,15 @@
# Hacker News 剪藏
打开 `https://news.ycombinator.com/item?id=<id>` 主帖详情页,点击插件的“剪藏当前网页”,或在页面上右键选择“保存 Hacker News 主帖到 EdgeEver”。工具栏操作优先保存选中文字;专用页面右键菜单保存主帖。
笔记保留页面提供的标题、作者、原始发布时间、剪藏时的分数、外部链接和主帖正文。支持 Ask HN、Show HN 和只有链接的主帖,不包含评论。完整主帖笔记带有 `web-clip` 和 `hacker-news` 标签。
外链文章仅记录链接,不抓取其正文。缺失的元数据直接省略;评论详情页、已删除主帖和无法读取的内容会被拒绝。剪藏期间页面切换会取消保存。不增加持久网站访问权限。
正文图片沿用现有附件流程。复制失败或达到数量限制时,正文仍保留,提示会说明部分图片仍使用外部链接;并发编辑不会被覆盖。
## 验证范围
自动验证使用脱敏的公开 HN DOM 样本,包括解析器测试、实际打包的后台与注入脚本测试,以及连接本地 API 并重新打开笔记的 Chromium 插件测试。图片存储用例在 HN 样本中加入合成图片。这些检查不能替代 Chrome、Edge、Firefox 的真实网站验收或商店发布。
[English](browser-clipper-hacker-news.md)
+126
View File
@@ -0,0 +1,126 @@
import { chromium, expect, test, type BrowserContext, type Worker } from "@playwright/test";
import { execFileSync } from "node:child_process";
import { cpSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
declare const chrome: any;
const repositoryRoot = resolve(fileURLToPath(new URL("../..", import.meta.url)));
const png = Buffer.from("iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=", "base64");
const platforms = [
{ name: "hacker-news", url: "https://news.ycombinator.com/item?id=40380738", title: "Show HN: I built a LLM-powered Ask HN: like Perplexity, but for HN comments", text: "Ollama on your machine!", excluded: "Matching the HN style" },
];
test("Hacker News clips persist through the extension and local API; archived images survive source blocking", async ({ request, baseURL }) => {
test.setTimeout(180_000);
execFileSync("bun", ["run", "build:extension"], { cwd: repositoryRoot, stdio: "pipe" });
const directory = mkdtempSync(join(tmpdir(), "edgeever-platform-extension-"));
const profile = mkdtempSync(join(tmpdir(), "edgeever-platform-browser-"));
cpSync(join(repositoryRoot, "apps/extension/dist"), directory, { recursive: true });
const manifestPath = join(directory, "manifest.json");
const manifest = JSON.parse(readFileSync(manifestPath, "utf8"));
// The test cannot grant activeTab by clicking browser chrome. Give only this
// temporary build the sampled hosts, image host and local instance access.
manifest.host_permissions = ["http://127.0.0.1/*", "https://news.ycombinator.com/*", "https://images.edgeever.test/*"];
writeFileSync(manifestPath, JSON.stringify(manifest));
let context: BrowserContext | undefined;
let tokenId = "";
const memoIds: string[] = [];
try {
const tokenResponse = await request.post("/api/v1/api-tokens", { data: {
name: "web-clipper-platform-e2e", scopes: ["read:memos", "write:memos", "read:resources", "write:resources", "read:notebooks"],
} });
expect(tokenResponse.status()).toBe(201);
const { token, apiToken } = await tokenResponse.json(); tokenId = apiToken.id;
context = await chromium.launchPersistentContext(profile, {
channel: "chromium", locale: "zh-CN", args: [`--disable-extensions-except=${directory}`, `--load-extension=${directory}`],
});
const worker: Worker = context.serviceWorkers()[0] ?? await context.waitForEvent("serviceworker");
await worker.evaluate((settings) => chrome.storage.local.set(settings), { instanceUrl: baseURL!, token, notebookId: "" });
const article = context.pages()[0] ?? await context.newPage();
let failure = false;
let sourceBlocked = false;
const sourceRequests: string[] = [];
await context.route(/^https:\/\//, async (route) => {
const url = route.request().url(); sourceRequests.push(url);
if (sourceBlocked) { await route.abort(); return; }
if (url.startsWith("https://images.edgeever.test/")) { await route.fulfill({ contentType: "image/png", body: png }); return; }
const platform = platforms.find((entry) => url === entry.url);
if (!platform) { await route.abort(); return; }
const html = failure ? "<html><body><h1>Sign in</h1><p>Unrelated comments</p></body></html>"
: readFileSync(join(repositoryRoot, `apps/extension/fixtures/platforms/${platform.name}.html`), "utf8")
.replace(/(<div class="toptext"[^>]*>)/, '$1<img src="https://images.edgeever.test/body.png">');
if (!failure) expect(html).toContain('src="https://images.edgeever.test/body.png"');
await route.fulfill({ contentType: "text/html; charset=utf-8", body: html });
});
const senderPromise = context.waitForEvent("page");
await worker.evaluate(() => chrome.windows.create({ url: chrome.runtime.getURL("options.html"), focused: false }));
const sender = await senderPromise; await sender.waitForLoadState();
const save = async (url: string) => {
await worker.evaluate(async (targetUrl) => {
const [tab] = await chrome.tabs.query({ url: targetUrl });
await chrome.tabs.update(tab.id, { active: true });
await chrome.windows.update(tab.windowId, { focused: true });
}, url);
return sender.evaluate(() => chrome.runtime.sendMessage({ type: "captureCurrentPage" }));
};
for (const platform of platforms) {
const before = (await (await request.get("/api/v1/memos?limit=100")).json()).memos as Array<{ id: string }>;
const oldIds = new Set(before.map((memo) => memo.id));
await article.goto(platform.url);
const saved = await save(platform.url);
expect(saved.ok).toBe(true);
const after = (await (await request.get("/api/v1/memos?limit=100")).json()).memos as Array<{ id: string; title: string }>;
const note = after.find((memo) => !oldIds.has(memo.id) && memo.title === platform.title);
expect(note).toBeTruthy(); memoIds.push(note!.id);
const detail = await request.get(`/api/v1/memos/${note!.id}`);
expect(detail.ok()).toBe(true);
const { memo } = await detail.json();
expect(memo.contentMarkdown).toContain(platform.text);
expect(memo.contentMarkdown).not.toContain(platform.excluded);
expect(memo.contentMarkdown).toContain(platform.url);
{
const resources = [...memo.contentMarkdown.matchAll(/!\[[^\]]*\]\(\/api\/v1\/resources\/([^/)]+)\/blob\)/g)] as RegExpMatchArray[];
expect(resources).toHaveLength(1);
await article.goto("about:blank"); sourceBlocked = true;
// With the source page gone and every external source host blocked,
// reopen the saved note's image from the EdgeEver resource endpoint.
const blob = await request.get(`/api/v1/resources/${resources[0][1]}/blob`);
expect(blob.ok()).toBe(true);
expect(Buffer.from(await blob.body()).equals(png)).toBe(true);
}
const preview = await context.newPage();
await preview.goto(baseURL!);
await preview.getByRole("button", { name: "全部笔记", exact: true }).click();
await preview.getByPlaceholder("搜索笔记").fill(platform.title);
await preview.locator(`[data-memo-id="${note!.id}"]`).locator("button").first().click();
const editor = preview.locator(".ProseMirror");
await expect(editor).toContainText(platform.text);
await expect(editor).not.toContainText(platform.excluded);
{
const image = editor.locator("img").first();
await expect(image).toBeVisible();
await expect.poll(() => image.evaluate((element) => element.complete && element.naturalWidth > 0)).toBe(true);
}
await preview.close();
sourceBlocked = false;
failure = true;
await article.goto(platform.url);
expect((await save(platform.url)).ok).toBe(false);
const unchanged = (await (await request.get("/api/v1/memos?limit=100")).json()).memos as Array<{ id: string }>;
expect(unchanged.filter((memo) => !oldIds.has(memo.id)).map((memo) => memo.id)).toEqual([note!.id]);
failure = false;
}
expect(sourceRequests.some((url) => /\.(mp3|m4a|mp4)(\?|$)/.test(url))).toBe(false);
} finally {
for (const id of memoIds) {
expect.soft((await request.delete(`/api/v1/memos/${id}`)).ok()).toBe(true);
expect.soft((await request.delete(`/api/v1/memos/${id}?permanent=1`)).ok()).toBe(true);
}
if (tokenId) await request.delete(`/api/v1/api-tokens/${tokenId}`);
await context?.close();
rmSync(directory, { recursive: true, force: true }); rmSync(profile, { recursive: true, force: true });
}
});