refactor: pipeline error propagation, me.yaml→me.ts, docs overhaul
Code:
- pipeline.ts: warn on stderr when step returns {error} object
- xiaohongshu/me.yaml → me.ts (evaluate code belongs in TS)
- index.ts: register xiaohongshu/me.ts
Docs:
- CLI-CREATOR: YAML vs TS decision tree, tap step docs with
real example, 'explore store first' debugging workflow
- SKILL.md: add tap to Pipeline Steps table, update XHS commands
- README + README.zh-CN: update xiaohongshu command list
This commit is contained in:
1 parent
f26b906bd5
commit
9df96db598
7 files changed
+140
-22
No files matched your search
+104
-7
@@ -106,6 +106,26 @@ opencli cascade https://api.example.com/hot
|
||||
|
||||
## Step 3: 编写适配器
|
||||
|
||||
### YAML vs TS?先看决策树
|
||||
|
||||
```
|
||||
你的 pipeline 里有 evaluate 步骤(内嵌 JS 代码)?
|
||||
→ ✅ 用 TypeScript (src/clis/<site>/<name>.ts),需在 index.ts 注册
|
||||
→ ❌ 纯声明式(navigate + tap + map + limit)?
|
||||
→ ✅ 用 YAML (src/clis/<site>/<name>.yaml),放入即自动注册
|
||||
```
|
||||
|
||||
| 场景 | 选择 | 示例 |
|
||||
|------|------|------|
|
||||
| 纯 fetch/select/map/limit | YAML | `v2ex/hot.yaml`, `hackernews/top.yaml` |
|
||||
| navigate + evaluate(fetch) + map | YAML(评估复杂度) | `zhihu/hot.yaml` |
|
||||
| navigate + tap + map | YAML ✅ | `xiaohongshu/feed.yaml`, `xiaohongshu/notifications.yaml` |
|
||||
| 有复杂 JS 逻辑(Pinia state 读取、条件分支) | TS | `xiaohongshu/me.ts`, `bilibili/me.ts` |
|
||||
| XHR 拦截 + 签名 | TS | `xiaohongshu/search.ts` |
|
||||
| GraphQL / 分页 / Wbi 签名 | TS | `bilibili/search.ts`, `twitter/search.ts` |
|
||||
|
||||
> **经验法则**:如果你发现 YAML 里嵌了超过 10 行 JS,改用 TS 更可维护。
|
||||
|
||||
### 方式 A: YAML Pipeline(声明式,推荐)
|
||||
|
||||
文件路径: `src/clis/<site>/<name>.yaml`,放入即自动注册。
|
||||
@@ -230,9 +250,65 @@ pipeline:
|
||||
columns: [rank, title, type, author, votes]
|
||||
```
|
||||
|
||||
#### Tier 4 — Store Action Bridge(`tap` 步骤,intercept 策略推荐)
|
||||
|
||||
适用于 Vue + Pinia/Vuex 的网站(如小红书),无须手动写 XHR 拦截代码:
|
||||
|
||||
```yaml
|
||||
# src/clis/xiaohongshu/notifications.yaml
|
||||
site: xiaohongshu
|
||||
name: notifications
|
||||
description: "小红书通知"
|
||||
domain: www.xiaohongshu.com
|
||||
strategy: intercept
|
||||
browser: true
|
||||
|
||||
args:
|
||||
type:
|
||||
type: str
|
||||
default: mentions
|
||||
description: "Notification type: mentions, likes, or connections"
|
||||
limit:
|
||||
type: int
|
||||
default: 20
|
||||
|
||||
columns: [rank, user, action, content, note, time]
|
||||
|
||||
pipeline:
|
||||
- navigate: https://www.xiaohongshu.com/notification
|
||||
- wait: 3
|
||||
- tap:
|
||||
store: notification # Pinia store name
|
||||
action: getNotification # Store action to call
|
||||
args: # Action arguments
|
||||
- ${{ args.type | default('mentions') }}
|
||||
capture: /you/ # URL pattern to capture response
|
||||
select: data.message_list # Extract sub-path from response
|
||||
timeout: 8
|
||||
- map:
|
||||
rank: ${{ index + 1 }}
|
||||
user: ${{ item.user_info.nickname }}
|
||||
action: ${{ item.title }}
|
||||
content: ${{ item.comment_info.content }}
|
||||
- limit: ${{ args.limit | default(20) }}
|
||||
```
|
||||
|
||||
> **`tap` 步骤自动完成**:注入 fetch+XHR 双拦截 → 查找 Pinia/Vuex store → 调用 action → 捕获匹配 URL 的响应 → 清理拦截。
|
||||
> 如果 store 或 action 找不到,会返回 `hint` 列出所有可用的 store actions,方便调试。
|
||||
|
||||
| tap 参数 | 必填 | 说明 |
|
||||
|---------|------|------|
|
||||
| `store` | ✅ | Pinia store 名称(如 `feed`, `search`, `notification`) |
|
||||
| `action` | ✅ | Store action 方法名 |
|
||||
| `capture` | ✅ | URL 子串匹配(匹配网络请求 URL) |
|
||||
| `args` | ❌ | 传给 action 的参数数组 |
|
||||
| `select` | ❌ | 从 captured JSON 中提取的路径(如 `data.items`) |
|
||||
| `timeout` | ❌ | 等待网络响应的超时秒数(默认 5s) |
|
||||
| `framework` | ❌ | `pinia` 或 `vuex`(默认自动检测) |
|
||||
|
||||
### 方式 B: TypeScript 适配器(编程式)
|
||||
|
||||
适用于需要 XHR 拦截、GraphQL、分页、复杂数据转换等场景。
|
||||
适用于需要嵌入 JS 代码读取 Pinia state、XHR 拦截、GraphQL、分页、复杂数据转换等场景。
|
||||
|
||||
文件路径: `src/clis/<site>/<name>.ts`,还需要在 `src/clis/index.ts` 中 import 注册。
|
||||
|
||||
@@ -379,24 +455,45 @@ opencli mysite hot --limit 3 -f json # JSON 输出确认字段完整
|
||||
|
||||
### tap 步骤调试(intercept 策略专用)
|
||||
|
||||
tap 步骤依赖正确的 Pinia store name 和 action name。如果不确定:
|
||||
> **⚠️ 不要猜 store name / action name**。先用 evaluate 探索,再写 YAML。
|
||||
|
||||
#### Step 1: 列出所有 Pinia store
|
||||
|
||||
在浏览器中打开目标网站后:
|
||||
|
||||
```bash
|
||||
# 运行 evaluate 探索 Pinia store 结构
|
||||
opencli evaluate "(() => {
|
||||
const app = document.querySelector('#app')?.__vue_app__;
|
||||
const pinia = app?.config?.globalProperties?.\$pinia;
|
||||
return [...pinia._s.keys()];
|
||||
})()"
|
||||
# 输出: ["user", "feed", "search", "notification", ...]
|
||||
```
|
||||
|
||||
tap 的错误提示会列出可用的 actions,例如:
|
||||
#### Step 2: 查看 store 的 action 名称
|
||||
|
||||
故意写一个错误 action 名,tap 会返回所有可用 actions:
|
||||
|
||||
```
|
||||
{ error: 'Action not found: fetchData on store user',
|
||||
hint: 'Available: getProfile, updateInfo, logout' }
|
||||
⚠ tap: Action not found: wrongName on store notification
|
||||
💡 Available: getNotification, replyComment, getNotificationCount, reset
|
||||
```
|
||||
|
||||
根据提示修正 action name 即可。
|
||||
#### Step 3: 用 network requests 确认 capture 模式
|
||||
|
||||
```bash
|
||||
# 在浏览器打开目标页面,查看网络请求
|
||||
# 找到目标 API 的 URL 特征(如 "/you/mentions"、"homefeed")
|
||||
```
|
||||
|
||||
#### 完整流程
|
||||
|
||||
```
|
||||
┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌────────┐
|
||||
│ 1. navigate │ ──▶ │ 2. 探索 store │ ──▶ │ 3. 写 YAML │ ──▶ │ 4. 测试 │
|
||||
│ 到目标页面 │ │ name/action │ │ tap 步骤 │ │ 运行验证 │
|
||||
└──────────────┘ └──────────────┘ └──────────────┘ └────────┘
|
||||
```
|
||||
|
||||
### Verbose 模式
|
||||
|
||||
|
||||
@@ -81,7 +81,7 @@ Public API commands (`hackernews`, `github search`, `v2ex`) need no browser at a
|
||||
|------|----------|------|
|
||||
| **bilibili** | `hot` `search` `me` `favorite` `history` `feed` `user-videos` | 🔐 Browser |
|
||||
| **zhihu** | `hot` `search` `question` | 🔐 Browser |
|
||||
| **xiaohongshu** | `search` `feed` | 🔐 Browser |
|
||||
| **xiaohongshu** | `search` `me` `notifications` `feed` | 🔐 Browser |
|
||||
| **twitter** | `trending` | 🔐 Browser |
|
||||
| **reddit** | `hot` | 🔐 Browser |
|
||||
| **github** | `trending` `search` | 🔐 / 🌐 |
|
||||
|
||||
+1
-1
@@ -81,7 +81,7 @@ npm install -g @jackwener/opencli@latest
|
||||
|------|------|------|
|
||||
| **bilibili** | `hot` `search` `me` `favorite` `history` `feed` `user-videos` | 🔐 浏览器 |
|
||||
| **zhihu** | `hot` `search` `question` | 🔐 浏览器 |
|
||||
| **xiaohongshu** | `search` `feed` | 🔐 浏览器 |
|
||||
| **xiaohongshu** | `search` `me` `notifications` `feed` | 🔐 浏览器 |
|
||||
| **twitter** | `trending` | 🔐 浏览器 |
|
||||
| **reddit** | `hot` | 🔐 浏览器 |
|
||||
| **github** | `trending` `search` | 🔐 / 🌐 |
|
||||
|
||||
@@ -55,6 +55,9 @@ opencli zhihu question --id 34816524 # 问题详情和回答
|
||||
|
||||
# 小红书 (browser)
|
||||
opencli xiaohongshu search --keyword "美食" # 搜索笔记
|
||||
opencli xiaohongshu me # 我的信息
|
||||
opencli xiaohongshu notifications # 通知(mentions/likes/connections)
|
||||
opencli xiaohongshu feed --limit 10 # 推荐 Feed
|
||||
|
||||
# GitHub (trending=browser, search=public)
|
||||
opencli github trending --limit 10 # GitHub Trending
|
||||
@@ -224,6 +227,7 @@ cli({
|
||||
| `sort` | Sort items | `sort: { by: score, order: desc }` |
|
||||
| `limit` | Cap result count | `limit: ${{ args.limit }}` |
|
||||
| `intercept` | Declarative XHR capture | `intercept: { trigger: "navigate:...", capture: "api/hot" }` |
|
||||
| `tap` | Store action + XHR capture | `tap: { store: "feed", action: "fetchFeeds", capture: "homefeed" }` |
|
||||
| `snapshot` | Page accessibility tree | `snapshot: { interactive: true }` |
|
||||
| `click` | Click element | `click: ${{ ref }}` |
|
||||
| `type` | Type text | `type: { ref: "@1", text: "hello" }` |
|
||||
|
||||
@@ -20,3 +20,4 @@ import './zhihu/question.js';
|
||||
|
||||
// xiaohongshu
|
||||
import './xiaohongshu/search.js';
|
||||
import './xiaohongshu/me.js';
|
||||
@@ -1,16 +1,21 @@
|
||||
site: xiaohongshu
|
||||
name: me
|
||||
description: "我的小红书个人信息"
|
||||
domain: www.xiaohongshu.com
|
||||
strategy: intercept
|
||||
browser: true
|
||||
/**
|
||||
* Xiaohongshu me — read self profile info from Pinia user store.
|
||||
*/
|
||||
import { cli, Strategy } from '../../registry.js';
|
||||
|
||||
columns: [nickname, red_id, fans, follows, notes, likes_collected]
|
||||
cli({
|
||||
site: 'xiaohongshu',
|
||||
name: 'me',
|
||||
description: '我的小红书个人信息',
|
||||
domain: 'www.xiaohongshu.com',
|
||||
strategy: Strategy.COOKIE,
|
||||
args: [],
|
||||
columns: ['nickname', 'red_id', 'fans', 'follows', 'notes', 'likes_collected'],
|
||||
func: async (page) => {
|
||||
await page.goto('https://www.xiaohongshu.com/user/profile/self');
|
||||
await page.wait(3);
|
||||
|
||||
pipeline:
|
||||
- navigate: https://www.xiaohongshu.com/user/profile/self
|
||||
- wait: 3
|
||||
- evaluate: |
|
||||
const data = await page.evaluate(`
|
||||
(() => {
|
||||
const app = document.querySelector('#app')?.__vue_app__;
|
||||
const pinia = app?.config?.globalProperties?.$pinia;
|
||||
@@ -19,12 +24,18 @@ pipeline:
|
||||
if (!userStore) return { error: 'User store not found' };
|
||||
const state = userStore.$state || {};
|
||||
const u = state.selfInfo || state.currentUserInfo || state.userInfo || {};
|
||||
return [{
|
||||
return {
|
||||
nickname: u.nickname || u.name || '',
|
||||
red_id: u.red_id || u.redId || '',
|
||||
fans: u.fans || u.fansCount || 0,
|
||||
follows: u.follows || u.followsCount || 0,
|
||||
notes: u.notes || u.noteCount || 0,
|
||||
likes_collected: u.collected || u.likedAndCollected || 0,
|
||||
}];
|
||||
};
|
||||
})()
|
||||
`);
|
||||
|
||||
if (!data || data.error) return [];
|
||||
return [data];
|
||||
},
|
||||
});
|
||||
@@ -26,6 +26,11 @@ export async function executePipeline(
|
||||
for (const [op, params] of Object.entries(step)) {
|
||||
if (debug) debugStepStart(i + 1, total, op, params);
|
||||
data = await executeStep(page, op, params, data, args);
|
||||
// Detect error objects returned by steps (e.g. tap store not found)
|
||||
if (data && typeof data === 'object' && !Array.isArray(data) && data.error) {
|
||||
process.stderr.write(` ${chalk.yellow('⚠')} ${chalk.yellow(op)}: ${data.error}\n`);
|
||||
if (data.hint) process.stderr.write(` ${chalk.dim('💡')} ${chalk.dim(data.hint)}\n`);
|
||||
}
|
||||
if (debug) debugStepResult(op, data);
|
||||
}
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user