Add Exa registration support and update README status

This commit is contained in:
skernelx committed 2026-03-17 23:01:13 +08:00
1 parent 715af4bb4b
commit efd14c6892
7 files changed
+375 -26

No files matched your search

+1
View File
@@ -11,6 +11,7 @@ ENV/
# 账号信息
accounts.txt
firecrawl_accounts.txt
exa_accounts.txt
accounts.csv
.env
.env.local
+26 -4
View File
@@ -4,9 +4,15 @@
这是当前的多服务注册器与聚合 API 上游工具,聚焦两件事:
- 注册 `Tavily` / `Firecrawl` key
- 注册 `Firecrawl` / `Exa` key
- 验证 key 是否真实可用,并把可用 key 提供给统一搜索层
## 当前状态
- `Firecrawl`:可用
- `Exa`:可用
- `Tavily`:按当前项目这轮本地测试,官网已关闭邮件注册入口,暂不可用
它不是 `MySearch` 的正式产品仓库。
如果你要部署公开可用的统一搜索控制台、MCP、Skill 和 Social / X
接入,请直接使用独立项目:
@@ -24,14 +30,14 @@
## Features
- 多服务启动台:启动时直接选择 Tavily 或 Firecrawl
- 多服务启动台:启动时可选择 Tavily / Firecrawl / Exa
- 自动环境准备:自动检查 `venv`、依赖和浏览器
- 邮箱链路统一:支持 Cloudflare Mail API 和 DuckMail
- 多域名可选:启动时选择本轮注册用的域名
- 并发注册:支持批量和并发执行
- 后台浏览器模式:默认 headless,必要时可切前台排查
- 真实可用性验证:拿到 key 后马上调用官方接口
- 自动上传到代理池:上传时会带上服务标记,服务端能自动识别 Tavily / Firecrawl 并写入各自池子
- 自动上传到代理池:上传时会带上服务标记,服务端能自动识别 Firecrawl / Exa / Tavily 并写入对应池子
- 适合作为统一搜索网关的上游:给 `MySearch-Proxy` 提供稳定的 key 来源
- 跨平台启动:Windows / macOS / Linux 都能直接跑
@@ -62,6 +68,8 @@ cp .env.example .env
编辑 `.env`,填好邮箱配置和可选上传配置。
如果已经配置了 `SERVER_URL` 和 `SERVER_ADMIN_PASSWORD`,注册成功后可以自动上传。
### 3. Run
macOS / Linux:
@@ -82,6 +90,20 @@ Windows:
start_auto.bat
```
启动后当前可选:
```text
1. Tavily
2. Firecrawl
3. Exa
```
补充说明:
- `Exa` 注册成功后会保存到 `exa_accounts.txt`
- `Exa` 的格式是 `email,EMAIL_OTP_ONLY,api_key`
- `Tavily` 选项目前仍保留在启动台里,但当前状态暂不可用
## 推荐搭配方式
如果你希望把注册出来的 key 接成公开可用的统一搜索服务,推荐直接搭配:
@@ -108,7 +130,7 @@ start_auto.bat
这里的 `tavily-key-generator` 只负责:
- 注册 Tavily / Firecrawl key
- 注册 Firecrawl / Exa key
- 验证 key 是否真实可用
- 提供可选聚合 API,供 `MySearch` / `MySearch-Proxy` 调用
+26 -4
View File
@@ -5,9 +5,15 @@
This repository is the multi-service registration toolkit and aggregation API
upstream for:
- registering `Tavily` / `Firecrawl` keys
- registering `Firecrawl` / `Exa` keys
- validating that the keys actually work before feeding a unified search layer
## Current Status
- `Firecrawl`: available
- `Exa`: available
- `Tavily`: based on the current local project testing, the official email registration entry is closed, so it is temporarily unavailable
It is not the main product repository for `MySearch`.
If you want the public-facing unified search console, MCP, Skill, and Social / X
integration, use the standalone project:
@@ -25,14 +31,14 @@ integration, use the standalone project:
## Features
- Multi-service launcher: choose Tavily or Firecrawl at startup
- Multi-service launcher: choose Tavily / Firecrawl / Exa at startup
- Automatic environment bootstrap: checks `venv`, dependencies, and browsers
- Unified mail layer: supports Cloudflare Mail API and DuckMail
- Multi-domain support: choose the active domain at runtime
- Concurrent registration: batch and parallel runs are supported
- Background browser mode: headless by default, visible mode when debugging
- Real usability verification: validates the key against the official API
- Automatic upload to proxy pools
- Automatic upload to proxy pools with a per-service marker
- A clean upstream for unified search gateways such as `MySearch-Proxy`
- Cross-platform startup: Windows, macOS, and Linux
@@ -63,6 +69,8 @@ cp .env.example .env
Edit `.env` and fill in your mail settings and optional upload settings.
If `SERVER_URL` and `SERVER_ADMIN_PASSWORD` are configured, successful registrations can be uploaded automatically.
### 3. Run
macOS / Linux:
@@ -83,6 +91,20 @@ Windows:
start_auto.bat
```
Current launcher options:
```text
1. Tavily
2. Firecrawl
3. Exa
```
Notes:
- `Exa` accounts are saved to `exa_accounts.txt`
- the saved format is `email,EMAIL_OTP_ONLY,api_key`
- the `Tavily` option is still shown in the launcher, but it is currently unavailable based on local testing
## Recommended Pairing
If you want to turn the generated keys into a public, reusable unified search
@@ -111,7 +133,7 @@ ships the runnable implementation.
This repo only handles:
- Tavily / Firecrawl key registration
- Firecrawl / Exa key registration
- live key validation
- optional aggregation API serving for `MySearch` / `MySearch-Proxy`
+244
View File
@@ -0,0 +1,244 @@
"""
使用 Camoufox 完成 Exa 注册
思路:通过邮箱验证码登录,跳过 onboarding,并提取默认 API Key
"""
import json
import os
import re
import threading
import time
import requests as std_requests
from camoufox.sync_api import Camoufox
from config import API_KEY_TIMEOUT, EMAIL_CODE_TIMEOUT, REGISTER_HEADLESS
from mail_provider import get_email_code
_HERE = os.path.dirname(os.path.abspath(__file__))
_SAVE_FILE = os.path.join(_HERE, "exa_accounts.txt")
_SAVE_LOCK = threading.Lock()
_ACCOUNT_PASSWORD_LABEL = "EMAIL_OTP_ONLY"
_EXA_AUTH_URL = "https://auth.exa.ai/?callbackUrl=https%3A%2F%2Fdashboard.exa.ai%2F"
_EXA_HOME_URL = "https://dashboard.exa.ai/home"
def fill_first_input(page, selectors, value):
"""填充第一个存在的输入框"""
for selector in selectors:
if page.query_selector(selector):
page.fill(selector, value)
return selector
return None
def click_first(page, selectors):
"""点击第一个存在的按钮/链接"""
for selector in selectors:
if page.query_selector(selector):
page.click(selector, no_wait_after=True)
return True
return False
def extract_api_key(page):
"""从页面文本或 HTML 中提取 Exa API Key。"""
patterns = []
try:
patterns.extend(re.findall(r"\b[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12}\b", page.locator("main").inner_text(), re.I))
except Exception:
pass
try:
patterns.extend(re.findall(r"\b[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12}\b", page.content(), re.I))
except Exception:
pass
for candidate in patterns:
return candidate
return None
def fetch_api_key_via_dashboard_api(page):
"""直接调用已登录 dashboard 的 get-api-keys 接口,优先拿完整 key。"""
try:
payload = page.evaluate(
"""
async () => {
const response = await fetch('/api/get-api-keys', {
method: 'GET',
credentials: 'include',
headers: {
'accept': 'application/json',
},
});
return {
status: response.status,
body: await response.text(),
};
}
"""
)
except Exception:
return None
if int(payload.get("status") or 0) != 200:
return None
try:
data = json.loads(payload.get("body") or "{}")
except Exception:
return None
for item in data.get("apiKeys", []):
candidate = (item.get("id") or "").strip()
if re.fullmatch(r"[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12}", candidate, re.I):
return candidate
return None
def ensure_dashboard_ready(page):
"""跳过 onboarding,落到可读 API Key 的 dashboard 页面。"""
if "dashboard.exa.ai" not in page.url.lower():
page.wait_for_url("**/dashboard.exa.ai/**", timeout=30000, wait_until="domcontentloaded")
if "/onboarding" in page.url.lower():
click_first(page, ['button:text-is("Skip")'])
time.sleep(1)
click_first(
page,
[
'button:text-is("Yes, I don\\\'t want the $10 in credits anyway!")',
'button:text-is("Yes")',
],
)
page.wait_for_url("**/dashboard.exa.ai/**", timeout=30000, wait_until="domcontentloaded")
time.sleep(2)
if "/home" not in page.url.lower():
page.goto(_EXA_HOME_URL, wait_until="networkidle", timeout=30000)
time.sleep(2)
def wait_for_api_key(page, timeout=20):
"""等待主页 API Key 卡片渲染并显示完整 key。"""
start_time = time.time()
while time.time() - start_time < timeout:
ensure_dashboard_ready(page)
api_key = fetch_api_key_via_dashboard_api(page)
if api_key:
return api_key
click_first(page, ['button:text-is("Show")'])
time.sleep(1)
api_key = extract_api_key(page)
if api_key:
return api_key
time.sleep(1)
return None
def save_account(email, api_key):
"""并发注册时串行写入 exa_accounts.txt"""
with _SAVE_LOCK:
with open(_SAVE_FILE, "a", encoding="utf-8") as file_obj:
file_obj.write(f"{email},{_ACCOUNT_PASSWORD_LABEL},{api_key}\n")
def verify_api_key(api_key, timeout=30):
"""真实调用 Exa API,验证新 key 可用"""
try:
response = std_requests.post(
"https://api.exa.ai/search",
json={
"query": "api key verification",
"numResults": 1,
},
headers={
"x-api-key": api_key,
"Content-Type": "application/json",
"Accept": "application/json",
},
timeout=timeout,
)
except Exception as exc:
print(f"❌ API Key 调用测试失败: {exc}")
return False
if response.status_code == 200:
print("✅ API Key 调用测试通过")
return True
preview = response.text.strip().replace("\n", " ")[:160]
print(f"❌ API Key 调用测试失败: HTTP {response.status_code}")
if preview:
print(f" 响应: {preview}")
return False
def register_with_browser(email, password):
"""使用浏览器完成 Exa 邮箱验证码注册"""
print(f"🌐 使用浏览器模式注册 Exa: {email}")
try:
with Camoufox(headless=REGISTER_HEADLESS) as browser:
page = browser.new_page()
page.goto(_EXA_AUTH_URL, wait_until="networkidle", timeout=30000)
time.sleep(2)
email_selector = fill_first_input(
page,
['input[type="email"]', 'input[placeholder="Email"]', 'input[aria-label="Email"]'],
email,
)
if not email_selector:
print("❌ Exa 登录页未找到邮箱输入框")
return None
if not click_first(page, ['button:text-is("Continue")']):
print("❌ Exa 登录页未找到 Continue 按钮")
return None
page.wait_for_selector('input[placeholder*="verification" i], input[aria-label*="verification" i]', timeout=30000)
print("✅ 到达 Exa 邮箱验证码页")
code = get_email_code(email, timeout=EMAIL_CODE_TIMEOUT, service="exa")
if not code:
return None
code_selector = fill_first_input(
page,
['input[placeholder*="verification" i]', 'input[aria-label*="verification" i]'],
code,
)
if not code_selector:
print("❌ Exa 验证码页未找到输入框")
return None
if not click_first(page, ['button:text-is("VERIFY CODE")', 'button:text-is("Verify Code")', 'button:text-is("Verify")']):
page.press(code_selector, "Enter")
page.wait_for_url("**/dashboard.exa.ai/**", timeout=30000, wait_until="domcontentloaded")
print("✅ Exa 登录成功")
api_key = wait_for_api_key(page, timeout=API_KEY_TIMEOUT)
if not api_key:
print("⚠️ 未找到 Exa API Key")
return None
print("🧪 验证 API Key 可用性...")
if not verify_api_key(api_key):
return None
save_account(email, api_key)
print("🎉 Exa 注册成功")
print(f" 邮箱: {email}")
print(f" 密码: {_ACCOUNT_PASSWORD_LABEL}")
print(f" Key : {api_key}")
return api_key
except Exception as exc:
print(f"❌ Exa 注册失败: {exc}")
return None
+19
View File
@@ -0,0 +1,19 @@
"""
Exa 注册统一入口
"""
from mail_provider import create_email
from exa_browser_solver import register_with_browser
def register(email, password):
"""统一注册入口"""
return register_with_browser(email, password)
if __name__ == "__main__":
email, password = create_email(service="exa")
result = register(email, password)
if result:
print(f"✅ 注册成功: {email}")
else:
print(f"❌ 注册失败: {email}")
+35 -12
View File
@@ -32,7 +32,7 @@ _DUCKMAIL_DOMAIN_PRIORITY = (
_DUCKMAIL_DOMAIN_CACHE = None
_DUCKMAIL_MAILBOX_CACHE = {}
_SELECTED_DOMAIN = ""
_SUPPORTED_SERVICES = ("tavily", "firecrawl")
_SUPPORTED_SERVICES = ("tavily", "firecrawl", "exa")
def rand_str(n=8):
@@ -72,7 +72,11 @@ def _normalize_service(service):
def _username_prefix(service):
service = _normalize_service(service)
return "fc" if service == "firecrawl" else "tavily"
if service == "firecrawl":
return "fc"
if service == "exa":
return "exa"
return "tavily"
def create_email(service="tavily"):
@@ -104,13 +108,13 @@ def get_verification_link(email, timeout=120):
)
def get_email_code(email, timeout=120):
def get_email_code(email, timeout=120, service="tavily"):
"""等待邮箱里的 6 位验证码。"""
print(f"📨 等待邮箱验证码(最多 {timeout} 秒)...")
return _poll_mailbox(
email=email,
timeout=timeout,
extractor=_extract_email_code,
extractor=lambda message: _extract_email_code(message, service=service),
found_message="✅ 收到 6 位验证码",
timeout_message="❌ 等待邮箱验证码超时",
error_prefix="读取邮箱验证码失败",
@@ -175,16 +179,35 @@ def _extract_verification_link(message):
return None
def _extract_email_code(message):
def _extract_email_code(message, service="tavily"):
service = _normalize_service(service)
subject = (message.get("subject") or "").lower()
if "verify your identity" not in subject and "verify" not in subject and "tavily" not in subject:
return None
text = message.get("text") or ""
content = _message_content(message)
match = re.search(r"\b(\d{6})\b", content)
if not match:
return None
return match.group(1)
combined = f"{subject}\n{content}".lower()
if service == "exa":
if "exa" not in combined:
return None
if "verification code" not in combined and "sign in" not in combined:
return None
for source in (text, content):
match = re.search(
r"verification code(?:\s+for\s+exa)?(?:\s+is)?[^0-9]*(\d{6})",
source,
re.IGNORECASE,
)
if match:
return match.group(1)
else:
if "verify your identity" not in subject and "verify" not in subject and "tavily" not in combined:
return None
for source in (text, content):
match = re.search(r"\b(\d{6})\b", source)
if match:
return match.group(1)
return None
def _iter_messages(email):
+24 -6
View File
@@ -125,6 +125,7 @@ from config import (
)
from tavily_core import create_email as create_tavily_email, register as register_tavily
from firecrawl_core import register as register_firecrawl
from exa_core import register as register_exa
from mail_provider import create_email, get_active_domain, get_configured_domains, set_selected_domain
# ──────────────────────────────────────────────
@@ -199,9 +200,21 @@ def validate_runtime_config(upload, show_provider_summary=True):
return True
def print_runtime_summary(service="tavily"):
service_name = "Tavily" if service == "tavily" else "Firecrawl"
output_file = "accounts.txt" if service == "tavily" else "firecrawl_accounts.txt"
account_prefix = "tavily-" if service == "tavily" else "fc-"
service_name = {
"tavily": "Tavily",
"firecrawl": "Firecrawl",
"exa": "Exa",
}.get(service, "Tavily")
output_file = {
"tavily": "accounts.txt",
"firecrawl": "firecrawl_accounts.txt",
"exa": "exa_accounts.txt",
}.get(service, "accounts.txt")
account_prefix = {
"tavily": "tavily-",
"firecrawl": "fc-",
"exa": "exa-",
}.get(service, "tavily-")
print(f"""
┌──────────────────────────────────────────┐
│ 多服务自动注册启动台 │
@@ -395,8 +408,10 @@ def register_one(index, total, upload, service="tavily"):
if service == "tavily":
result = register_tavily(email, password)
else: # firecrawl
elif service == "firecrawl":
result = register_firecrawl(email, password)
else:
result = register_exa(email, password)
if result and result != "SUCCESS_NO_KEY":
if upload:
@@ -476,14 +491,17 @@ def prompt_service_choice():
print("\n请选择要注册的服务:")
print(" 1. Tavily")
print(" 2. Firecrawl")
print(" 3. Exa")
while True:
print("请输入选项 (1-2,默认 1): ", end="")
print("请输入选项 (1-3,默认 1): ", end="")
raw = input().strip()
if raw == "" or raw == "1":
return "tavily"
elif raw == "2":
return "firecrawl"
elif raw == "3":
return "exa"
else:
print("❌ 请输入有效编号")
continue
@@ -492,7 +510,7 @@ def main():
service = prompt_service_choice()
print_runtime_summary(service)
# Firecrawl 不需要 Solver
# 目前只有 Tavily 需要 Solver
need_solver = (service == "tavily")
if not validate_runtime_config(False, show_provider_summary=True):