diff --git a/docs/handover/assistant-workspace.md b/docs/handover/assistant-workspace.md index 88ad90b..11dc510 100644 --- a/docs/handover/assistant-workspace.md +++ b/docs/handover/assistant-workspace.md @@ -2,41 +2,118 @@ ## 概述 -助理工作区让用户指定一个目录存放 AI 人格/记忆文件（soul.md、user.md、memory.md、claude.md），在助理项目会话中自动注入系统提示词。附带对话式引导问卷（10 题）和每日轻量问询（3 题）。 +助理工作区让用户指定一个目录存放 AI 人格/记忆文件（soul.md、user.md、memory.md、claude.md），在助理项目会话中自动注入系统提示词。附带对话式引导问卷（13 题）和每日轻量问询（3 题）。V2 新增索引/检索/分类/组织能力。 ## 目录结构 ``` workspace/ -├── claude.md # 执行规则 -├── soul.md # 人格风格 -├── user.md # 用户画像 -├── memory.md # 长期记忆 +├── claude.md # 执行规则 +├── soul.md # 人格风格 +├── user.md # 用户画像 +├── memory.md # 长期记忆（append-only，去重保护） +├── README.ai.md # 自动生成的目录概览 +├── PATH.ai.md # 自动生成的路径索引 +├── Inbox/ # 默认捕获目录 +├── memory/ +│ └── daily/ # 每日记忆（episodic，按日期命名） +│ ├── 2026-03-05.md +│ └── archive.md # 过期日记的摘要归档 └── .assistant/ - └── state.json # 状态持久化 + ├── state.json # 状态持久化（schemaVersion: 2） + ├── config.json # 工作区配置（组织方式、归档策略、索引参数） + ├── taxonomy.json # 分类体系（目录角色推断 + 演化建议） + └── index/ + ├── manifest.jsonl # 文件级元数据索引 + ├── chunks.jsonl # Markdown 分块索引 + └── hotset.json # 高频访问文件（pinned + frequent） ``` ## 核心模块 -### `src/lib/assistant-workspace.ts` +### `src/lib/assistant-workspace.ts`（V1 + V2） | 函数 | 职责 | |------|------| | `validateWorkspace(dir)` | 检查目录和文件状态 | -| `initializeWorkspace(dir)` | 补建缺失模板 + state.json | -| `loadWorkspaceFiles(dir)` | 读 4 文件，每文件上限 8000 chars | -| `assembleWorkspacePrompt(files)` | XML 标签拼接，总上限 40000 chars | +| `initializeWorkspace(dir)` | 补建缺失模板 + state.json + V2 目录 | +| `loadWorkspaceFiles(dir)` | 读核心文件 + 每日记忆 + 根文档 | +| `assembleWorkspacePrompt(files, retrievalResults?)` | XML 标签拼接，按预算分配（总上限 40000 chars） | | `loadState(dir)` / `saveState(dir, state)` | 读写 `.assistant/state.json` | -| `needsDailyCheckIn(state)` | 判断是否需要每日问询（前置检查 onboarding 完成） | +| `needsDailyCheckIn(state)` | 判断是否需要每日问询 | | `generateDirectoryDocs(dir)` | 扫描子目录生成 README.ai.md + PATH.ai.md | +| `generateRootDocs(dir)` | 生成根级 README.ai.md + PATH.ai.md | +| `ensureDailyDir(dir)` | 确保 `memory/daily/` 目录存在 | +| `writeDailyMemory(dir, date, content)` | 写每日记忆文件 | +| `loadDailyMemories(dir, count)` | 加载最近 N 天的每日记忆 | +| `migrateStateV1ToV2(dir)` | V1→V2 自动迁移 | -文件兼容映射：lowercase canonical → uppercase fallback（如 `soul.md` → `Soul.md` → `SOUL.md`）。 +### `src/lib/workspace-config.ts`（V2 新增） + +工作区配置管理。`loadConfig()` 读取 `.assistant/config.json`，与 `DEFAULT_CONFIG` 深合并。配置项包括： + +- `organizationStyle`：project / time / topic / mixed +- `captureDefault`：新信息默认存放目录 +- `archivePolicy`：归档天数、是否自动归档 +- `ignore`：索引忽略的 glob 模式 +- `index`：分块大小、最大深度、文件大小上限 + +### `src/lib/workspace-taxonomy.ts`（V2 新增） + +分类体系推断。从目录名映射已知角色（notes/project/journal/archive/inbox 等），按 confidence 评分。`classifyPath()` 按最长路径前缀匹配分类。 + +### `src/lib/workspace-indexer.ts`（V2 新增） + +Markdown 索引引擎。 + +| 函数 | 职责 | +|------|------| +| `indexWorkspace(dir, { force? })` | **增量**索引：比较 mtime 跳过未变文件，仅重索引 stale 文件 | +| `indexFile(dir, filePath)` | 单文件索引：frontmatter 解析 + 分块 + 分类 | +| `chunkMarkdown(content, size, overlap)` | 按 heading 切 section，大 section 滑窗分块 | +| `extractMarkdownMeta(content)` | 提取 title/tags/headings/aliases | +| `loadManifest(dir)` / `loadChunks(dir)` | 读取 JSONL 索引文件 | +| `getIndexStats(dir)` | 统计文件数/分块数/stale 数 | + +**增量索引机制**：每次聊天开始时自动调用（在检索之前），对未变文件复用已有 manifest/chunks 条目，仅写入有变化时才更新文件。UI "重新索引"按钮走 `force: true` 全量模式。 + +### `src/lib/workspace-retrieval.ts`（V2 新增） + +关键词检索引擎。 + +| 函数 | 职责 | +|------|------| +| `parseQuery(query)` | 分词：英文按空白/标点 + 停用词过滤；**CJK 文本按 bigram + unigram** | +| `scoreManifest(entry, keywords)` | 标题 10 / 标签 8 / 别名 7 / 标题 5 / 路径 3 | +| `scoreChunk(chunk, keywords)` | 正文出现次数 + heading 2x | +| `searchWorkspace(dir, query, { limit })` | 综合评分 + **hotset 加权**（pinned +5，frequent +log2(count)） | +| `loadHotset(dir)` / `updateHotset(dir, paths)` | 高频文件管理 | + +**CJK 支持**：`parseQuery()` 检测 CJK 字符范围（U+4E00–U+9FFF 等），CJK 连续 run 生成 unigram + bigram，英文 run 按空白切分 + 停用词过滤。确保中文查询能命中标题/标签/正文中的中文内容。 + +### `src/lib/workspace-organizer.ts`（V2 新增） + +| 函数 | 职责 | +|------|------| +| `assertContained(dir, relativePath)` | 路径边界校验：拒绝绝对路径、`~`、`../` 穿越、符号链接逃逸 | +| `captureNote(dir, title, content)` | 将笔记写入默认捕获目录（经 `assertContained` 校验） | +| `classifyAndSuggest(dir, filePath)` | 基于 taxonomy 建议分类和目标路径 | +| `moveFile(dir, from, to)` | 移动文件（经 `assertContained` 校验） | +| `archiveDailyMemories(dir)` | 超过 retention 天数的日记摘要归档 + 删除原文件 | +| `promoteDailyToLongTerm(dir, date)` | 提取 "Candidate Long-Term Memory" 追加到 memory.md（**幂等**） | +| `suggestTaxonomyEvolution(dir)` | 分析文件分布，建议新建/归档分类 | + +**memory.md 去重保护**： +- `promoteDailyToLongTerm()` 检查日记文件的 `` 标记 + memory.md 的 `## Promoted from {date}` 标题，双重防重复 +- check-in 路由的 inline promotion 也检查 memory.md 中是否已存在同日标题 ### Prompt 注入（`src/app/api/chat/route.ts`） 仅当 `session.working_directory === getSetting('assistant_workspace_path')` 时注入： -1. 工作区文件内容（`` XML） -2. 引导/问询指令（``） +1. **增量索引**（在检索前执行，确保当前轮拿到最新内容） +2. **检索**：对用户消息调用 `searchWorkspace()` 取相关上下文 + 更新 hotset +3. **工作区文件内容**（`` XML，含核心文件 + 每日记忆 + 检索结果） +4. **引导/问询指令**（``） 普通项目聊天完全不受影响。 @@ -44,27 +121,44 @@ workspace/ 1. **Mount 触发**：`ChatView` mount 时 500ms 延迟检测，使用 `startStream()` 直接发送（非 `sendMessage`） 2. **Focus 兜底**：`MessageInput` onFocus 首次触发，防 mount 未生效 -3. **防重复**：`assistantTriggerFiredRef`（组件级 ref）防止同一 mount 周期内重复触发；`state.hookTriggeredSessionId` 按 session 粒度防止跨页面重复 +3. **防重复**：`assistantTriggerFiredRef`（组件级 ref）防止同一 mount 周期内重复触发；`state.hookTriggeredSessionId` 按 session 粒度防止跨页面重复（但若 session 无消息则视为上次失败，允许重试） 4. **autoTrigger 标志**：触发时携带 `autoTrigger: true`，后端跳过保存用户消息和标题更新，实现"AI 先说话"体验 5. **hookTriggeredSessionId 清理**：引导/问询完成后前端调用 `POST /api/workspace/hook-triggered` 发送 `{ sessionId: '__clear__' }` 清除标记，确保下次进入可再次触发 6. **最新会话校验**：每日问询仅在该工作区最新会话中触发，旧会话打开不会劫持问询（通过 `GET /api/workspace/latest-session` 校验） ### 确定性落盘 -AI 在对话中输出 `onboarding-complete` / `checkin-complete` 代码块 → 前端 `detectAssistantCompletion()` 解析 → 调 POST `/api/workspace/onboarding` 或 `/api/workspace/checkin` → 后端 AI 生成文件内容（失败回退原始答案）→ 清除 `hookTriggeredSessionId`。 +AI 在对话中输出 `onboarding-complete` / `checkin-complete` 代码块 → 前端 `detectAssistantCompletion()` 解析 → 调 POST 对应端点 → 后端 AI 生成文件内容（失败回退原始答案）→ 清除 `hookTriggeredSessionId`。 + +**安全绑定**：`detectAssistantCompletion()` 在处理 fence 前先校验 `workingDirectory === assistant_workspace_path`，确保普通项目聊天或旧助理会话即使输出同样的 fence 也不会写入工作区文件。前端同时把 `sessionId` 传给后端，后端再次校验 `session.working_directory === workspacePath`（双重校验）。 + +**注意**：`onboarding-complete` / `checkin-complete` 是自定义 fence 语言标记，不是真正的编程语言。Shiki / Streamdown 渲染时会降级为纯文本（已通过 `supportsLanguage()` 前置检查静默处理）。 + +### 每日 check-in 数据流（V2 改进） + +1. 用户回答 3 个问题 → 前端解析 `checkin-complete` JSON +2. 后端并行调 AI 生成：每日记忆条目 + 长期记忆提升候选 + user.md 更新 +3. 每日记忆写入 `memory/daily/{date}.md`（episodic，不破坏长期记忆） +4. 长期记忆提升：仅追加新的稳定事实到 memory.md（日期去重） +5. 自动归档：过期日记摘要 → `archive.md`，7 天前的日记尝试 promote（幂等） ## API 端点 | 路由 | 方法 | 职责 | |------|------|------| -| `/api/settings/workspace` | GET | 返回路径 + 文件状态 + state | -| `/api/settings/workspace` | PUT | 保存路径，可选初始化 | -| `/api/workspace/session` | POST | 创建或复用助理会话（onboarding 新建，checkin 复用） | -| `/api/workspace/onboarding` | POST | 接收答案 → AI 生成 soul.md + user.md | -| `/api/workspace/checkin` | POST | 接收答案 → AI 更新 memory.md + user.md | -| `/api/workspace/hook-triggered` | POST | 设置/清除 hookTriggeredSessionId（`__clear__` 哨兵值清除） | +| `/api/settings/workspace` | GET | 返回路径 + 文件状态 + state + **taxonomy**；路径无效时返回 `valid: false` + `reason`（path_not_found / not_a_directory / not_readable / not_writable） | +| `/api/settings/workspace` | PUT | 原子保存路径：先执行 initialize / resetOnboarding 副作用，全部成功后才 `setSetting()`；支持 `{ path, initialize?, resetOnboarding? }` | +| `/api/workspace/inspect` | GET | 切换前预检：`?path=xxx` → 返回 exists / isDirectory / readable / writable / hasAssistantData / workspaceStatus（empty / normal_directory / existing_workspace / partial_workspace / invalid）+ summary | +| `/api/workspace/session` | POST | 创建或复用助理会话（onboarding 新建，checkin 复用）；创建前校验路径存在性 + R_OK + W_OK | +| `/api/workspace/onboarding` | POST | 接收 13 题答案 + sessionId → AI 生成 soul.md + user.md + claude.md + memory.md + config.json + taxonomy.json；校验 session 归属 | +| `/api/workspace/checkin` | POST | 接收答案 + sessionId → 写每日记忆 + 增量更新 memory.md + user.md；校验 session 归属 | +| `/api/workspace/hook-triggered` | POST | 设置/清除 hookTriggeredSessionId | | `/api/workspace/latest-session` | GET | 返回指定 workingDirectory 的最新会话 ID | -| `/api/workspace/docs` | POST | 刷新目录文档 | +| `/api/workspace/docs` | POST | 刷新根目录 + 子目录文档（`generateRootDocs` + `generateDirectoryDocs`） | +| `/api/workspace/index` | GET | 返回索引统计（文件数/分块数/stale 数） | +| `/api/workspace/index` | POST | **强制全量重建索引** | +| `/api/workspace/search` | GET | 关键词检索（`?q=xxx&limit=5`） | +| `/api/workspace/organize` | POST | 组织操作：capture / classify / move / archive / suggest-evolution；路径参数强制相对路径 + containment 校验 | ## 状态字段（`.assistant/state.json`） @@ -72,16 +166,92 @@ AI 在对话中输出 `onboarding-complete` / `checkin-complete` 代码块 → interface AssistantWorkspaceState { onboardingComplete: boolean; lastCheckInDate: string | null; // "YYYY-MM-DD" - schemaVersion: number; // 当前 1 + schemaVersion: number; // 当前 2 hookTriggeredSessionId?: string; // 防重复触发 } ``` +## 工作区切换流程 + +切换助理工作区路径是一个多步流程，确保数据安全和用户确认： + +### 1. 路径输入与实时校验 + +- 输入框 debounce 500ms 后触发路径格式校验（绿 ✓ / 红 ✕ / 加载 spinner） +- 不存在的路径视为"可创建"，不阻断保存 + +### 2. Inspect 预检 + +点击保存时，前端先调 `GET /api/workspace/inspect?path=xxx`，获取目标路径的 `workspaceStatus`： + +| 状态 | 含义 | 确认弹窗行为 | +|------|------|-------------| +| `not_found` | 路径不存在 | 提示"将创建目录并初始化"→ `executeSave(initialize=true)` | +| `empty` | 空目录 | 提示"将初始化为工作区"→ `executeSave(initialize=true)` | +| `normal_directory` | 非空普通目录 | 提示"将在此初始化工作区"→ `executeSave(initialize=true)` | +| `existing_workspace` | 已有完整工作区 | 提供"继续接管"（reuse 模式）或"重新引导"（reset onboarding） | +| `partial_workspace` | 有 .assistant 但缺文件 | 提供"继续接管"或"重新初始化" | + +### 3. 原子保存 + +`PUT /api/settings/workspace` 先执行所有副作用（`initializeWorkspace()` / `saveState(onboardingComplete: false)`），全部成功后才调 `setSetting()` 持久化路径。任一步骤失败则路径设置不变。 + +### 4. 自动导航 + +保存成功后： +- 前端根据 `navigateMode` 创建会话（`'new'` → onboarding 模式新建，`'reuse'` → checkin 模式复用最新会话） +- 派发 `assistant-workspace-switched` 自定义事件 +- 自动跳转到新会话 + +### 5. 切换横幅 + +`ChatView` 监听 `assistant-workspace-switched` 事件，并检测当前会话的 `workingDirectory` 是否与 `assistant_workspace_path` 一致。不一致时显示黄色横幅，提供"打开新助理项目"按钮。 + +### 无效路径兜底 + +GET settings 返回 `valid: false` + `reason` 时，设置页面隐藏引导/问询/tab 面板，显示红色警告横幅。用户需修改路径后才能继续。 + +## 设置 UI（`AssistantWorkspaceSection.tsx`） + +底部 tab 面板（仅当 `workspace.valid !== false` 时显示）： + +| Tab | 内容 | +|-----|------| +| 文件状态 | 4 个核心文件存在/大小 + 刷新目录文档按钮 | +| 分类体系 | 从 taxonomy.json 读取的分类列表 | +| 文件索引 | 索引统计（文件数/分块数/过期数/上次索引时间）+ 重新索引按钮 | +| 组织管理 | 归档旧记忆按钮 | + ## 关键约束 - 引导与问询互斥：onboarding 完成当天不触发 daily check-in - 每日问询复用最后一条会话，不新建 -- `hookTriggeredSessionId` 按 session 粒度防重复，换 session 可重新触发；完成后自动清除 +- `hookTriggeredSessionId` 按 session 粒度防重复，换 session 可重新触发；完成后自动清除；若 session 无消息（上次触发失败）则允许重试 - `autoTrigger` 贯穿 ChatView → stream-session-manager → chat/route.ts，跳过用户消息保存和标题更新 - 每日问询不受 `messages.length > 0` 限制，可在有历史消息的复用会话中触发 - 文件截断策略：每文件 8000 chars（head 6000 + truncated marker + tail 1800） +- **索引在检索前执行**：确保当前轮对话能看到最新文件内容 +- **增量索引**：聊天入口的索引调用仅处理 stale 文件，大文库不会拖慢首轮响应 +- **memory.md 不可被覆写**：所有写入都是 append，promotion 有日期去重保护 +- **hotset 影响排序**：pinned 文件 +5 分，frequent 文件 +log2(count) 分（上限 +4） +- **路径校验 R_OK + W_OK**：GET settings、session 创建均要求读写权限，只读目录返回 `not_writable` 错误 +- **切换原子性**：PUT settings 先执行副作用再持久化，失败不留脏状态 +- **切换横幅**：`ChatView` 通过 `assistant-workspace-switched` 事件 + workingDirectory 比对检测不匹配；仅对旧助理会话显示（普通项目聊天不受影响）；事件监听器匹配 `oldPath`，挂载时通过 inspect 检查 `hasAssistantData` +- **Completion fence 安全绑定**：前端校验 `workingDirectory === assistant_workspace_path` + 后端校验 `session.working_directory === workspacePath`，双重防止非助理会话写入工作区 +- **Organize 路径边界**：`assertContained()` 拒绝绝对路径 / `~` / `../` / 符号链接逃逸；API 入口 `validateRelativePath()` 预检；`captureDefault` 写入 config 时强制相对路径 +- **每日问询入口**：设置页中仅当 `onboardingComplete === true` 时显示每日问询卡片 +- **刷新文档**：`/api/workspace/docs` 同时调用 `generateRootDocs()` + `generateDirectoryDocs()` + +## V1 → V2 变更摘要 + +| 维度 | V1 | V2 | +|------|----|----| +| Schema | `schemaVersion: 1` | `schemaVersion: 2`，自动迁移 | +| 引导问卷 | 10 题 | 13 题（新增组织方式、默认收件箱、归档策略） | +| 引导产物 | soul.md + user.md | + claude.md + memory.md + config.json + taxonomy.json | +| 记忆模型 | 每日 check-in 覆写 memory.md | 每日记忆写入 `memory/daily/` + memory.md 仅追加 | +| 索引 | 无 | Markdown 分块索引（manifest + chunks JSONL） | +| 检索 | 无 | 关键词检索 + CJK bigram + hotset 加权 | +| 分类 | 无 | 目录角色推断 + 演化建议 | +| 组织 | 无 | 捕获/分类/移动/归档/长期记忆晋升 | +| 设置 UI | 文件状态 | + 分类体系 / 文件索引 / 组织管理 tab | diff --git a/package-lock.json b/package-lock.json index 4ea28f9..3de1b63 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "codepilot", - "version": "0.26.0", + "version": "0.27.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "codepilot", - "version": "0.26.0", + "version": "0.27.0", "dependencies": { "@ai-sdk/anthropic": "^3.0.47", "@ai-sdk/google": "^3.0.31", diff --git a/package.json b/package.json index bb2194e..6b1c0aa 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "codepilot", - "version": "0.26.0", + "version": "0.27.0", "private": true, "author": { "name": "op7418", diff --git a/src/__tests__/unit/assistant-workspace.test.ts b/src/__tests__/unit/assistant-workspace.test.ts index 62b6b01..58ae2d4 100644 --- a/src/__tests__/unit/assistant-workspace.test.ts +++ b/src/__tests__/unit/assistant-workspace.test.ts @@ -8,6 +8,7 @@ * 2. Input focus fallback: hookTriggeredSessionId prevents repeat * 3. Daily check-in: needsDailyCheckIn respects onboarding state * 4. Workspace prompt scoping: only assistant project sessions get prompts + * 5. V2: Daily memory write/load, v1→v2 migration, budget-aware prompt assembly */ import { describe, it, beforeEach, afterEach } from 'node:test'; @@ -29,6 +30,11 @@ const { loadWorkspaceFiles, assembleWorkspacePrompt, generateDirectoryDocs, + ensureDailyDir, + writeDailyMemory, + loadDailyMemories, + migrateStateV1ToV2, + generateRootDocs, } = require('../../lib/assistant-workspace') as typeof import('../../lib/assistant-workspace'); const { createSession, getLatestSessionByWorkingDirectory, closeDb } = require('../../lib/db') as typeof import('../../lib/db'); @@ -53,7 +59,7 @@ describe('Assistant Workspace', () => { const state = JSON.parse(fs.readFileSync(statePath, 'utf-8')); assert.equal(state.onboardingComplete, false); assert.equal(state.lastCheckInDate, null); - assert.equal(state.schemaVersion, 1); + assert.equal(state.schemaVersion, 2); }); it('should create all 4 template files', () => { @@ -63,6 +69,12 @@ describe('Assistant Workspace', () => { assert.ok(fs.existsSync(path.join(workDir, 'user.md'))); assert.ok(fs.existsSync(path.join(workDir, 'memory.md'))); }); + + it('should create V2 directories (memory/daily, Inbox)', () => { + initializeWorkspace(workDir); + assert.ok(fs.existsSync(path.join(workDir, 'memory', 'daily'))); + assert.ok(fs.existsSync(path.join(workDir, 'Inbox'))); + }); }); describe('onboarding auto-trigger detection', () => { @@ -111,26 +123,24 @@ describe('Assistant Workspace', () => { describe('daily check-in respects onboarding state', () => { it('should not trigger check-in if onboarding not complete', () => { - const state = { onboardingComplete: false, lastCheckInDate: null, schemaVersion: 1 }; + const state = { onboardingComplete: false, lastCheckInDate: null, schemaVersion: 2 }; assert.equal(needsDailyCheckIn(state), false); }); it('should trigger check-in if onboarding done and no check-in today', () => { - const state = { onboardingComplete: true, lastCheckInDate: '2020-01-01', schemaVersion: 1 }; + const state = { onboardingComplete: true, lastCheckInDate: '2020-01-01', schemaVersion: 2 }; assert.equal(needsDailyCheckIn(state), true); }); it('should not trigger check-in if already done today', () => { const today = new Date().toISOString().slice(0, 10); - const state = { onboardingComplete: true, lastCheckInDate: today, schemaVersion: 1 }; + const state = { onboardingComplete: true, lastCheckInDate: today, schemaVersion: 2 }; assert.equal(needsDailyCheckIn(state), false); }); it('onboarding day should skip daily check-in (lastCheckInDate set)', () => { - // Simulates what happens after onboarding completes: - // onboardingComplete=true, lastCheckInDate=today const today = new Date().toISOString().slice(0, 10); - const state = { onboardingComplete: true, lastCheckInDate: today, schemaVersion: 1 }; + const state = { onboardingComplete: true, lastCheckInDate: today, schemaVersion: 2 }; assert.equal(needsDailyCheckIn(state), false); }); }); @@ -201,6 +211,298 @@ describe('Assistant Workspace', () => { assert.ok(pathContent.includes('Path Index')); }); }); + + // ===== V2 Tests ===== + + describe('daily memory write and load', () => { + it('should write and read daily memory', () => { + initializeWorkspace(workDir); + const today = '2024-03-06'; + const content = '# Work Log\nDid some coding.'; + + writeDailyMemory(workDir, today, content); + + const dailyPath = path.join(workDir, 'memory', 'daily', `${today}.md`); + assert.ok(fs.existsSync(dailyPath), 'Daily memory file should exist'); + + const read = fs.readFileSync(dailyPath, 'utf-8'); + assert.equal(read, content); + }); + + it('should load most recent daily memories', () => { + initializeWorkspace(workDir); + writeDailyMemory(workDir, '2024-03-04', 'Day 1'); + writeDailyMemory(workDir, '2024-03-05', 'Day 2'); + writeDailyMemory(workDir, '2024-03-06', 'Day 3'); + + const memories = loadDailyMemories(workDir, 2); + assert.equal(memories.length, 2); + assert.equal(memories[0].date, '2024-03-06'); + assert.equal(memories[1].date, '2024-03-05'); + }); + + it('should return empty array for no daily memories', () => { + initializeWorkspace(workDir); + const memories = loadDailyMemories(workDir, 2); + assert.equal(memories.length, 0); + }); + }); + + describe('v1 to v2 migration', () => { + it('should migrate v1 state to v2', () => { + // Create a v1-style workspace + const stateDir = path.join(workDir, '.assistant'); + fs.mkdirSync(stateDir, { recursive: true }); + fs.writeFileSync( + path.join(stateDir, 'state.json'), + JSON.stringify({ onboardingComplete: true, lastCheckInDate: '2024-01-01', schemaVersion: 1 }), + 'utf-8' + ); + + migrateStateV1ToV2(workDir); + + const state = loadState(workDir); + assert.equal(state.schemaVersion, 2); + assert.ok(fs.existsSync(path.join(workDir, 'memory', 'daily'))); + assert.ok(fs.existsSync(path.join(workDir, 'Inbox'))); + }); + + it('should not re-migrate v2 state', () => { + initializeWorkspace(workDir); + const state = loadState(workDir); + assert.equal(state.schemaVersion, 2); + + // Should not throw or change anything + migrateStateV1ToV2(workDir); + const reloaded = loadState(workDir); + assert.equal(reloaded.schemaVersion, 2); + }); + }); + + describe('budget-aware prompt assembly', () => { + it('should include daily memories in prompt', () => { + initializeWorkspace(workDir); + fs.writeFileSync(path.join(workDir, 'soul.md'), '# Soul\nI am helpful.', 'utf-8'); + writeDailyMemory(workDir, '2024-03-06', '# Today\nDid coding.'); + + const files = loadWorkspaceFiles(workDir); + const prompt = assembleWorkspacePrompt(files); + + assert.ok(prompt.includes('')); + assert.ok(prompt.includes('I am helpful')); + assert.ok(prompt.includes('Did coding')); + }); + + it('should include retrieval results in prompt', () => { + initializeWorkspace(workDir); + fs.writeFileSync(path.join(workDir, 'soul.md'), '# Soul\nI am helpful.', 'utf-8'); + + const files = loadWorkspaceFiles(workDir); + const results = [ + { path: 'notes/test.md', heading: 'Test', snippet: 'Some test content', score: 15, source: 'title' as const }, + ]; + const prompt = assembleWorkspacePrompt(files, results); + + assert.ok(prompt.includes('retrieval-result')); + assert.ok(prompt.includes('Some test content')); + }); + + it('should respect total prompt limit', () => { + initializeWorkspace(workDir); + // Write large content to test truncation + const largeContent = '# Soul\n' + 'x'.repeat(50000); + fs.writeFileSync(path.join(workDir, 'soul.md'), largeContent, 'utf-8'); + + const files = loadWorkspaceFiles(workDir); + const prompt = assembleWorkspacePrompt(files); + + assert.ok(prompt.length <= 45000, 'Prompt should not exceed budget (with some overhead)'); + }); + }); + + describe('root docs generation', () => { + it('should generate README.ai.md and PATH.ai.md at root', () => { + initializeWorkspace(workDir); + // Create some subdirs + fs.mkdirSync(path.join(workDir, 'notes')); + fs.mkdirSync(path.join(workDir, 'projects')); + + const generated = generateRootDocs(workDir); + assert.ok(generated.length >= 2); + assert.ok(fs.existsSync(path.join(workDir, 'README.ai.md'))); + assert.ok(fs.existsSync(path.join(workDir, 'PATH.ai.md'))); + }); + + it('should include root docs in loaded files', () => { + initializeWorkspace(workDir); + fs.mkdirSync(path.join(workDir, 'notes')); + generateRootDocs(workDir); + + const files = loadWorkspaceFiles(workDir); + assert.ok(files.rootReadme, 'Should have rootReadme'); + assert.ok(files.rootPath, 'Should have rootPath'); + }); + }); +}); + +// --------------------------------------------------------------------------- +// Tests for review fixes +// --------------------------------------------------------------------------- + +describe('parseQuery CJK support', () => { + const { parseQuery } = require('../../lib/workspace-retrieval') as typeof import('../../lib/workspace-retrieval'); + + it('should tokenize Chinese text into bigrams and unigrams', () => { + const tokens = parseQuery('项目排期'); + // Should contain individual chars and bigrams + assert.ok(tokens.includes('项'), 'should include unigram 项'); + assert.ok(tokens.includes('目'), 'should include unigram 目'); + assert.ok(tokens.includes('项目'), 'should include bigram 项目'); + assert.ok(tokens.includes('排期'), 'should include bigram 排期'); + }); + + it('should handle mixed Chinese and English', () => { + const tokens = parseQuery('下周project排期'); + assert.ok(tokens.includes('下周'), 'should include CJK bigram'); + assert.ok(tokens.includes('project'), 'should include English word'); + assert.ok(tokens.includes('排期'), 'should include CJK bigram'); + }); + + it('should still filter English stop words', () => { + const tokens = parseQuery('the project is good'); + assert.ok(!tokens.includes('the')); + assert.ok(!tokens.includes('is')); + assert.ok(tokens.includes('project')); + assert.ok(tokens.includes('good')); + }); +}); + +describe('incremental indexWorkspace', () => { + const { indexWorkspace, loadManifest } = require('../../lib/workspace-indexer') as typeof import('../../lib/workspace-indexer'); + let wsDir: string; + + beforeEach(() => { + wsDir = fs.mkdtempSync(path.join(os.tmpdir(), 'idx-test-')); + fs.writeFileSync(path.join(wsDir, 'note1.md'), '# Note 1\nHello', 'utf-8'); + fs.writeFileSync(path.join(wsDir, 'note2.md'), '# Note 2\nWorld', 'utf-8'); + }); + + afterEach(() => { + fs.rmSync(wsDir, { recursive: true, force: true }); + }); + + it('should skip unchanged files on second run', () => { + const first = indexWorkspace(wsDir); + assert.equal(first.fileCount, 2); + + // Second run: nothing changed, should reuse existing + const second = indexWorkspace(wsDir); + assert.equal(second.fileCount, 2); + + // Manifest should be identical + const manifest = loadManifest(wsDir); + assert.equal(manifest.length, 2); + }); + + it('should re-index only modified files', () => { + indexWorkspace(wsDir); + const manifestBefore = loadManifest(wsDir); + const note1Before = manifestBefore.find(m => m.path === 'note1.md')!; + + // Modify note1 (ensure mtime changes) + const futureTime = Date.now() + 1000; + fs.writeFileSync(path.join(wsDir, 'note1.md'), '# Note 1 Updated\nChanged content', 'utf-8'); + fs.utimesSync(path.join(wsDir, 'note1.md'), new Date(futureTime), new Date(futureTime)); + + indexWorkspace(wsDir); + const manifestAfter = loadManifest(wsDir); + + const note1After = manifestAfter.find(m => m.path === 'note1.md')!; + const note2After = manifestAfter.find(m => m.path === 'note2.md')!; + const note2Before = manifestBefore.find(m => m.path === 'note2.md')!; + + // note1 should have changed hash + assert.notEqual(note1After.hash, note1Before.hash); + // note2 should be unchanged + assert.equal(note2After.hash, note2Before.hash); + }); + + it('force mode should re-index all files', () => { + indexWorkspace(wsDir); + const result = indexWorkspace(wsDir, { force: true }); + assert.equal(result.fileCount, 2); + }); +}); + +describe('memory.md promotion dedup', () => { + const { promoteDailyToLongTerm } = require('../../lib/workspace-organizer') as typeof import('../../lib/workspace-organizer'); + let wsDir: string; + + beforeEach(() => { + wsDir = fs.mkdtempSync(path.join(os.tmpdir(), 'promo-test-')); + fs.mkdirSync(path.join(wsDir, 'memory', 'daily'), { recursive: true }); + fs.writeFileSync(path.join(wsDir, 'memory.md'), '# Memory\n', 'utf-8'); + }); + + afterEach(() => { + fs.rmSync(wsDir, { recursive: true, force: true }); + }); + + it('should promote content on first call', () => { + const dailyContent = '## Work Log\nDid stuff\n\n## Candidate Long-Term Memory\nUser prefers dark mode and uses Vim keybindings daily.\n'; + fs.writeFileSync(path.join(wsDir, 'memory', 'daily', '2026-02-20.md'), dailyContent, 'utf-8'); + + const result = promoteDailyToLongTerm(wsDir, '2026-02-20'); + assert.equal(result, true); + + const memory = fs.readFileSync(path.join(wsDir, 'memory.md'), 'utf-8'); + assert.ok(memory.includes('Promoted from 2026-02-20')); + }); + + it('should NOT promote same date twice (idempotent)', () => { + const dailyContent = '## Candidate Long-Term Memory\nUser prefers dark mode and uses Vim keybindings daily.\n'; + fs.writeFileSync(path.join(wsDir, 'memory', 'daily', '2026-02-20.md'), dailyContent, 'utf-8'); + + promoteDailyToLongTerm(wsDir, '2026-02-20'); + const memoryAfterFirst = fs.readFileSync(path.join(wsDir, 'memory.md'), 'utf-8'); + + // Second call should return false + const result = promoteDailyToLongTerm(wsDir, '2026-02-20'); + assert.equal(result, false); + + const memoryAfterSecond = fs.readFileSync(path.join(wsDir, 'memory.md'), 'utf-8'); + assert.equal(memoryAfterFirst, memoryAfterSecond, 'memory.md should not change on second call'); + }); +}); + +describe('hotset boosts search results', () => { + const { indexWorkspace } = require('../../lib/workspace-indexer') as typeof import('../../lib/workspace-indexer'); + const { searchWorkspace, updateHotset } = require('../../lib/workspace-retrieval') as typeof import('../../lib/workspace-retrieval'); + let wsDir: string; + + beforeEach(() => { + wsDir = fs.mkdtempSync(path.join(os.tmpdir(), 'hotset-test-')); + // Two notes with similar relevance to "design" + fs.writeFileSync(path.join(wsDir, 'alpha.md'), '# Design Notes\nSome design thoughts', 'utf-8'); + fs.writeFileSync(path.join(wsDir, 'beta.md'), '# Design Patterns\nSome design patterns', 'utf-8'); + indexWorkspace(wsDir, { force: true }); + }); + + afterEach(() => { + fs.rmSync(wsDir, { recursive: true, force: true }); + }); + + it('should boost frequently accessed files in search ranking', () => { + // Access beta many times to build frequency + for (let i = 0; i < 10; i++) { + updateHotset(wsDir, ['beta.md']); + } + + const results = searchWorkspace(wsDir, 'design'); + assert.ok(results.length >= 2, 'Should find both files'); + // beta.md should be boosted to top due to hotset frequency + assert.equal(results[0].path, 'beta.md', 'Frequently accessed file should rank higher'); + }); }); // Clean up DB diff --git a/src/app/api/chat/route.ts b/src/app/api/chat/route.ts index acc67b4..b5dcc5a 100644 --- a/src/app/api/chat/route.ts +++ b/src/app/api/chat/route.ts @@ -180,8 +180,31 @@ export async function POST(request: NextRequest) { const isAssistantProject = sessionWd === workspacePath; if (isAssistantProject) { + // Incremental reindex BEFORE search so current turn sees latest content + try { + const { indexWorkspace } = await import('@/lib/workspace-indexer'); + indexWorkspace(workspacePath); + } catch { + // indexer not available, skip + } + const files = loadWorkspaceFiles(workspacePath); - workspacePrompt = assembleWorkspacePrompt(files); + + // Retrieval: search workspace index for relevant context + let retrievalResults: import('@/types').SearchResult[] | undefined; + try { + const { searchWorkspace, updateHotset } = await import('@/lib/workspace-retrieval'); + if (content.length > 10) { + retrievalResults = searchWorkspace(workspacePath, content, { limit: 5 }); + if (retrievalResults.length > 0) { + updateHotset(workspacePath, retrievalResults.map(r => r.path)); + } + } + } catch { + // retrieval module not available, skip + } + + workspacePrompt = assembleWorkspacePrompt(files, retrievalResults); const state = loadState(workspacePath); @@ -190,7 +213,7 @@ export async function POST(request: NextRequest) { assistantProjectInstructions = ` You are now in the assistant workspace onboarding session. Your task is to interview the user to build their profile. -Ask the following 10 questions ONE AT A TIME. Wait for the user's answer before asking the next question. Be conversational and friendly. +Ask the following 13 questions ONE AT A TIME. Wait for the user's answer before asking the next question. Be conversational and friendly. 1. How should I address you? 2. What name should I use for myself? @@ -202,14 +225,17 @@ Ask the following 10 questions ONE AT A TIME. Wait for the user's answer before 8. What information may be written to long-term memory? 9. What information must never be written to long-term memory? 10. What three things should I do first when entering a project? +11. How do you organize your materials? (by project / time / topic / mixed) +12. Where should new information go by default? +13. How should completed tasks be archived? After all questions are answered, output a summary of the collected answers in exactly this format — this is critical for the system to process your answers: \`\`\`onboarding-complete -{"q1":"answer1","q2":"answer2","q3":"answer3","q4":"answer4","q5":"answer5","q6":"answer6","q7":"answer7","q8":"answer8","q9":"answer9","q10":"answer10"} +{"q1":"answer1","q2":"answer2","q3":"answer3","q4":"answer4","q5":"answer5","q6":"answer6","q7":"answer7","q8":"answer8","q9":"answer9","q10":"answer10","q11":"answer11","q12":"answer12","q13":"answer13"} \`\`\` -Do NOT try to write files yourself. The system will automatically generate soul.md and user.md from your collected answers. +Do NOT try to write files yourself. The system will automatically generate soul.md, user.md, claude.md, memory.md, config.json, and taxonomy.json from your collected answers. Start by greeting the user and asking the first question. `; @@ -228,11 +254,12 @@ After collecting all 3 answers, output a summary in exactly this format: {"q1":"answer1","q2":"answer2","q3":"answer3"} \`\`\` -Do NOT try to write files yourself. The system will automatically update memory.md and user.md from your collected answers. +Do NOT try to write files yourself. The system will automatically write a daily memory entry and update user.md from your collected answers. Start by greeting the user and asking the first question. `; } + } } } catch (e) { diff --git a/src/app/api/settings/workspace/route.ts b/src/app/api/settings/workspace/route.ts index 42c1d35..076a7b9 100644 --- a/src/app/api/settings/workspace/route.ts +++ b/src/app/api/settings/workspace/route.ts @@ -1,13 +1,48 @@ import { NextRequest, NextResponse } from 'next/server'; import fs from 'fs'; import { getSetting, setSetting } from '@/lib/db'; -import { validateWorkspace, initializeWorkspace, loadState } from '@/lib/assistant-workspace'; +import { validateWorkspace, initializeWorkspace, loadState, saveState } from '@/lib/assistant-workspace'; export async function GET() { try { const workspacePath = getSetting('assistant_workspace_path'); if (!workspacePath) { - return NextResponse.json({ path: null, files: {}, state: null }); + return NextResponse.json({ path: null, valid: false, reason: 'no_path_configured', files: {}, state: null }); + } + + // Check if the path is actually valid before proceeding + let pathExists = false; + let isDirectory = false; + let readable = false; + try { + const stat = fs.statSync(workspacePath); + pathExists = true; + isDirectory = stat.isDirectory(); + } catch { /* path doesn't exist */ } + + let writable = false; + if (pathExists && isDirectory) { + try { + fs.accessSync(workspacePath, fs.constants.R_OK); + readable = true; + } catch { /* not readable */ } + try { + fs.accessSync(workspacePath, fs.constants.W_OK); + writable = true; + } catch { /* not writable */ } + } + + if (!pathExists) { + return NextResponse.json({ path: workspacePath, valid: false, reason: 'path_not_found', files: {}, state: null }); + } + if (!isDirectory) { + return NextResponse.json({ path: workspacePath, valid: false, reason: 'not_a_directory', files: {}, state: null }); + } + if (!readable) { + return NextResponse.json({ path: workspacePath, valid: false, reason: 'not_readable', files: {}, state: null }); + } + if (!writable) { + return NextResponse.json({ path: workspacePath, valid: false, reason: 'not_writable', files: {}, state: null }); } const validation = validateWorkspace(workspacePath); @@ -30,11 +65,30 @@ export async function GET() { }; } + // Load taxonomy for UI + let taxonomy: Array<{ id: string; label: string; role: string; confidence: number; source: string; paths: string[] }> = []; + try { + const { loadTaxonomy } = await import('@/lib/workspace-taxonomy'); + const taxData = loadTaxonomy(workspacePath); + taxonomy = taxData.categories.map(c => ({ + id: c.id, + label: c.label, + role: c.role, + confidence: c.confidence, + source: c.source, + paths: c.paths, + })); + } catch { + // taxonomy module not available + } + return NextResponse.json({ path: workspacePath, + valid: true, exists: validation.exists, files: fileStatus, state, + taxonomy, }); } catch (e) { console.error('[settings/workspace] GET failed:', e); @@ -45,19 +99,83 @@ export async function GET() { export async function PUT(request: NextRequest) { try { const body = await request.json(); - const { path: workspacePath, initialize } = body as { path: string; initialize?: boolean }; + const { path: workspacePath, initialize, resetOnboarding } = body as { path: string; initialize?: boolean; resetOnboarding?: boolean }; if (!workspacePath || typeof workspacePath !== 'string') { return NextResponse.json({ error: 'Invalid workspace path' }, { status: 400 }); } - setSetting('assistant_workspace_path', workspacePath); + // Validate the path before saving + let pathExists = false; + let isDirectory = false; + try { + const stat = fs.statSync(workspacePath); + pathExists = true; + isDirectory = stat.isDirectory(); + } catch { /* path doesn't exist */ } + if (!pathExists) { + // Only create directory tree when user explicitly requests initialization + if (!initialize) { + return NextResponse.json( + { error: 'Path does not exist. Set initialize=true to create it.', code: 'path_not_found' }, + { status: 400 } + ); + } + // Create the directory for initialization + try { + fs.mkdirSync(workspacePath, { recursive: true }); + } catch (mkdirErr) { + return NextResponse.json( + { error: `Failed to create directory: ${mkdirErr instanceof Error ? mkdirErr.message : 'unknown error'}`, code: 'mkdir_failed' }, + { status: 400 } + ); + } + } else if (!isDirectory) { + return NextResponse.json( + { error: 'Path exists but is not a directory.', code: 'not_a_directory' }, + { status: 400 } + ); + } + + // Check read/write permissions (on existing or newly created directory) + try { + fs.accessSync(workspacePath, fs.constants.R_OK | fs.constants.W_OK); + } catch { + return NextResponse.json( + { error: 'Directory is not readable/writable.', code: 'permission_denied' }, + { status: 400 } + ); + } + + // Perform initialization/reset BEFORE saving the setting (atomic: if init fails, setting is unchanged) let createdFiles: string[] = []; if (initialize) { - createdFiles = initializeWorkspace(workspacePath); + try { + createdFiles = initializeWorkspace(workspacePath); + } catch (initErr) { + return NextResponse.json( + { error: `Failed to initialize workspace: ${initErr instanceof Error ? initErr.message : 'unknown error'}`, code: 'init_failed' }, + { status: 500 } + ); + } } + if (resetOnboarding) { + try { + const state = loadState(workspacePath); + saveState(workspacePath, { ...state, onboardingComplete: false }); + } catch (resetErr) { + return NextResponse.json( + { error: `Failed to reset onboarding: ${resetErr instanceof Error ? resetErr.message : 'unknown error'}`, code: 'reset_failed' }, + { status: 500 } + ); + } + } + + // All side-effects succeeded — now commit the setting + setSetting('assistant_workspace_path', workspacePath); + return NextResponse.json({ success: true, createdFiles }); } catch (e) { console.error('[settings/workspace] PUT failed:', e); diff --git a/src/app/api/workspace/checkin/route.ts b/src/app/api/workspace/checkin/route.ts index 0146a83..0c96207 100644 --- a/src/app/api/workspace/checkin/route.ts +++ b/src/app/api/workspace/checkin/route.ts @@ -1,8 +1,8 @@ import { NextRequest, NextResponse } from 'next/server'; import fs from 'fs'; import path from 'path'; -import { getSetting, getDefaultProviderId } from '@/lib/db'; -import { loadState, saveState } from '@/lib/assistant-workspace'; +import { getSetting, getDefaultProviderId, getSession } from '@/lib/db'; +import { loadState, saveState, writeDailyMemory } from '@/lib/assistant-workspace'; import { generateTextFromProvider } from '@/lib/text-generator'; const CHECK_IN_QUESTIONS = [ @@ -35,12 +35,20 @@ export async function POST(request: NextRequest) { } const body = await request.json(); - const { answers } = body as { answers: Record }; + const { answers, sessionId } = body as { answers: Record; sessionId?: string }; if (!answers || typeof answers !== 'object') { return NextResponse.json({ error: 'Invalid answers format' }, { status: 400 }); } + // Validate that the calling session belongs to this workspace + if (sessionId) { + const session = getSession(sessionId); + if (session && session.working_directory !== workspacePath) { + return NextResponse.json({ error: 'Session does not belong to current workspace' }, { status: 403 }); + } + } + const qaText = CHECK_IN_LABELS.map((q, i) => { const key = `q${i + 1}`; return `Q: ${q}\nA: ${answers[key] || '(skipped)'}`; @@ -60,27 +68,98 @@ export async function POST(request: NextRequest) { const providerId = getDefaultProviderId() || ''; const model = getSetting('default_model') || 'claude-sonnet-4-20250514'; - const memoryPrompt = `You maintain a memory.md file for an AI assistant. Given the user's daily check-in answers and the existing memory file, generate an UPDATED memory.md. Add new facts, update changed information, remove outdated items. Keep it organized with markdown headers. Keep under 4000 characters.\n\nExisting memory.md:\n${existingMemory || '(empty)'}\n\nToday's check-in (${today}):\n${qaText}`; + // Generate daily memory entry (episodic, not destructive) + const dailyMemoryPrompt = `You maintain daily memory entries for an AI assistant. Given the user's daily check-in answers, generate a daily memory entry for ${today}. - const userPrompt = `You maintain a user.md profile for an AI assistant. Given the user's daily check-in answers and the existing profile, generate an UPDATED user.md. Only update sections affected by today's answers. Keep it organized with markdown headers. Keep under 2000 characters.\n\nExisting user.md:\n${existingUser || '(empty)'}\n\nToday's check-in (${today}):\n${qaText}`; +Format it with these sections: +## Work Log +(What the user accomplished today) - const [newMemory, newUser] = await Promise.all([ - generateTextFromProvider({ providerId, model, system: 'You maintain knowledge files for AI assistants. Output only the file content, no explanations.', prompt: memoryPrompt }), +## Priority Changes +(Any shifts in goals or priorities) + +## To Remember +(Things the user wants remembered) + +## Candidate Long-Term Memory +(Facts that seem stable enough to promote to long-term memory — only include genuinely persistent facts, not transient updates) + +Keep under 2000 characters. + +Today's check-in (${today}): +${qaText}`; + + // Generate promotion candidates: only stable facts to append to memory.md + const promotionPrompt = `You maintain a long-term memory file for an AI assistant. Given the user's check-in and the existing memory, output ONLY new stable facts that should be APPENDED to memory.md. These must be genuinely persistent facts (user preferences, recurring patterns, important relationships), NOT daily transients. + +If there's nothing worth promoting, output exactly: (nothing to promote) + +Existing memory.md: +${existingMemory || '(empty)'} + +Today's check-in (${today}): +${qaText}`; + + // Generate incremental user.md updates + const userPrompt = `You maintain a user.md profile for an AI assistant. Given the user's daily check-in answers and the existing profile, generate an UPDATED user.md. Only update sections affected by today's answers. Keep it organized with markdown headers. Keep under 2000 characters. + +Existing user.md: +${existingUser || '(empty)'} + +Today's check-in (${today}): +${qaText}`; + + const [dailyContent, promotionContent, newUser] = await Promise.all([ + generateTextFromProvider({ providerId, model, system: 'You maintain knowledge files for AI assistants. Output only the file content, no explanations.', prompt: dailyMemoryPrompt }), + generateTextFromProvider({ providerId, model, system: 'You maintain knowledge files for AI assistants. Output only the content to append, no explanations.', prompt: promotionPrompt }), generateTextFromProvider({ providerId, model, system: 'You maintain user profile documents. Output only the file content, no explanations.', prompt: userPrompt }), ]); - if (!newMemory.trim()) { - throw new Error('AI returned empty memory content'); + // Write daily memory file (episodic, per-day) + if (dailyContent.trim()) { + writeDailyMemory(workspacePath, today, dailyContent); } - fs.writeFileSync(memoryPath, newMemory, 'utf-8'); + + // Promote stable facts to memory.md (additive append, NOT rewrite) + // Dedup: skip if today's promotion already exists in memory.md + if (promotionContent.trim() && !promotionContent.includes('(nothing to promote)')) { + const currentMemory = fs.existsSync(memoryPath) ? fs.readFileSync(memoryPath, 'utf-8') : ''; + if (!currentMemory.includes(`## Promoted from ${today}`)) { + const appendText = `\n\n## Promoted from ${today}\n${promotionContent.trim()}\n`; + fs.appendFileSync(memoryPath, appendText, 'utf-8'); + } + } + + // Incremental user.md update if (existingUser && newUser.trim()) { fs.writeFileSync(userPath, newUser, 'utf-8'); } + + // Archive old daily memories and promote candidates + try { + const { archiveDailyMemories, promoteDailyToLongTerm } = await import('@/lib/workspace-organizer'); + archiveDailyMemories(workspacePath); + + // Promote memories that are 7+ days old + const sevenDaysAgo = new Date(); + sevenDaysAgo.setDate(sevenDaysAgo.getDate() - 7); + const dailyDir = path.join(workspacePath, 'memory', 'daily'); + if (fs.existsSync(dailyDir)) { + const oldFiles = fs.readdirSync(dailyDir) + .filter(f => /^\d{4}-\d{2}-\d{2}\.md$/.test(f)) + .filter(f => f.replace('.md', '') <= sevenDaysAgo.toISOString().slice(0, 10)); + for (const f of oldFiles) { + promoteDailyToLongTerm(workspacePath, f.replace('.md', '')); + } + } + } catch { + // organizer module not available, skip archival + } } catch (e) { - console.warn('[workspace/checkin] AI generation failed, appending raw answers:', e); - // Fallback: append raw answers to memory.md - const appendText = `\n\n## Check-in ${today}\n${qaText}\n`; - fs.appendFileSync(memoryPath, appendText, 'utf-8'); + console.warn('[workspace/checkin] AI generation failed, writing raw daily entry:', e); + // Fallback: write raw answers as daily memory + const rawDailyContent = `# Daily Check-in ${today}\n\n${qaText}\n`; + writeDailyMemory(workspacePath, today, rawDailyContent); } // Update state diff --git a/src/app/api/workspace/docs/route.ts b/src/app/api/workspace/docs/route.ts index c015aa4..3db0515 100644 --- a/src/app/api/workspace/docs/route.ts +++ b/src/app/api/workspace/docs/route.ts @@ -1,6 +1,6 @@ import { NextResponse } from 'next/server'; import { getSetting } from '@/lib/db'; -import { generateDirectoryDocs } from '@/lib/assistant-workspace'; +import { generateDirectoryDocs, generateRootDocs } from '@/lib/assistant-workspace'; export async function POST() { try { @@ -12,12 +12,13 @@ export async function POST() { ); } - const files = generateDirectoryDocs(workspacePath); - return NextResponse.json({ files }); + const rootFiles = generateRootDocs(workspacePath); + const directoryFiles = generateDirectoryDocs(workspacePath); + return NextResponse.json({ root: rootFiles, directory: directoryFiles, files: [...rootFiles, ...directoryFiles] }); } catch (e) { - console.error('[workspace/docs] Failed to generate directory docs:', e); + console.error('[workspace/docs] Failed to generate docs:', e); return NextResponse.json( - { error: 'Failed to generate directory docs' }, + { error: 'Failed to generate docs' }, { status: 500 }, ); } diff --git a/src/app/api/workspace/index/route.ts b/src/app/api/workspace/index/route.ts new file mode 100644 index 0000000..aebad8e --- /dev/null +++ b/src/app/api/workspace/index/route.ts @@ -0,0 +1,36 @@ +import { NextResponse } from 'next/server'; +import { getSetting } from '@/lib/db'; + +export async function GET() { + try { + const workspacePath = getSetting('assistant_workspace_path'); + if (!workspacePath) { + return NextResponse.json({ error: 'No workspace path configured' }, { status: 400 }); + } + + const { getIndexStats } = await import('@/lib/workspace-indexer'); + const stats = getIndexStats(workspacePath); + + return NextResponse.json(stats); + } catch (e) { + console.error('[workspace/index] GET failed:', e); + return NextResponse.json({ error: 'Failed to get index stats' }, { status: 500 }); + } +} + +export async function POST() { + try { + const workspacePath = getSetting('assistant_workspace_path'); + if (!workspacePath) { + return NextResponse.json({ error: 'No workspace path configured' }, { status: 400 }); + } + + const { indexWorkspace } = await import('@/lib/workspace-indexer'); + const result = indexWorkspace(workspacePath, { force: true }); + + return NextResponse.json({ success: true, ...result }); + } catch (e) { + console.error('[workspace/index] POST failed:', e); + return NextResponse.json({ error: 'Indexing failed' }, { status: 500 }); + } +} diff --git a/src/app/api/workspace/inspect/route.ts b/src/app/api/workspace/inspect/route.ts new file mode 100644 index 0000000..654e2bb --- /dev/null +++ b/src/app/api/workspace/inspect/route.ts @@ -0,0 +1,144 @@ +import { NextRequest, NextResponse } from 'next/server'; +import fs from 'fs'; +import path from 'path'; + +type WorkspaceStatus = 'empty' | 'normal_directory' | 'existing_workspace' | 'partial_workspace' | 'invalid'; + +const ASSISTANT_FILES = ['soul.md', 'Soul.md', 'SOUL.md', 'user.md', 'User.md', 'USER.md', 'PROFILE.md', 'memory.md', 'Memory.md', 'MEMORY.md', 'claude.md', 'Claude.md', 'CLAUDE.md', 'AGENTS.md']; +const CORE_FILE_KEYS = ['soul', 'user', 'memory', 'claude'] as const; + +function countExistingCoreFiles(dir: string): number { + const found = new Set(); + const keyMap: Record = { + 'soul.md': 'soul', 'Soul.md': 'soul', 'SOUL.md': 'soul', + 'user.md': 'user', 'User.md': 'user', 'USER.md': 'user', 'PROFILE.md': 'user', + 'memory.md': 'memory', 'Memory.md': 'memory', 'MEMORY.md': 'memory', + 'claude.md': 'claude', 'Claude.md': 'claude', 'CLAUDE.md': 'claude', 'AGENTS.md': 'claude', + }; + for (const filename of ASSISTANT_FILES) { + if (fs.existsSync(path.join(dir, filename))) { + const key = keyMap[filename]; + if (key) found.add(key); + } + } + return found.size; +} + +export async function GET(request: NextRequest) { + try { + const { searchParams } = new URL(request.url); + const inspectPath = searchParams.get('path'); + + if (!inspectPath || typeof inspectPath !== 'string') { + return NextResponse.json({ error: 'Missing required query parameter: path' }, { status: 400 }); + } + + // Check existence + let exists = false; + let isDirectory = false; + try { + const stat = fs.statSync(inspectPath); + exists = true; + isDirectory = stat.isDirectory(); + } catch { + // Path doesn't exist + } + + if (!exists || !isDirectory) { + return NextResponse.json({ + exists, + isDirectory, + readable: false, + writable: false, + hasAssistantData: false, + workspaceStatus: 'invalid' as WorkspaceStatus, + }); + } + + // Check permissions + let readable = false; + let writable = false; + try { + fs.accessSync(inspectPath, fs.constants.R_OK); + readable = true; + } catch { /* not readable */ } + try { + fs.accessSync(inspectPath, fs.constants.W_OK); + writable = true; + } catch { /* not writable */ } + + if (!readable) { + return NextResponse.json({ + exists, + isDirectory, + readable, + writable, + hasAssistantData: false, + workspaceStatus: 'invalid' as WorkspaceStatus, + }); + } + + // Check for .assistant/state.json + const statePath = path.join(inspectPath, '.assistant', 'state.json'); + const hasAssistantData = fs.existsSync(statePath); + + // Determine workspace status + let workspaceStatus: WorkspaceStatus; + let entries: fs.Dirent[]; + try { + entries = fs.readdirSync(inspectPath, { withFileTypes: true }); + } catch { + entries = []; + } + + // Filter out hidden entries for "emptiness" check + const visibleEntries = entries.filter(e => !e.name.startsWith('.')); + const hasAssistantDir = fs.existsSync(path.join(inspectPath, '.assistant')); + const fileCount = countExistingCoreFiles(inspectPath); + + if (hasAssistantData) { + workspaceStatus = 'existing_workspace'; + } else if (hasAssistantDir || fileCount > 0) { + // Has some assistant-related files but no state.json + workspaceStatus = 'partial_workspace'; + } else if (visibleEntries.length === 0) { + workspaceStatus = 'empty'; + } else { + workspaceStatus = 'normal_directory'; + } + + // Build response + const result: Record = { + exists, + isDirectory, + readable, + writable, + hasAssistantData, + workspaceStatus, + }; + + // If existing_workspace, return summary + if (workspaceStatus === 'existing_workspace') { + try { + const raw = fs.readFileSync(statePath, 'utf-8'); + const state = JSON.parse(raw) as { onboardingComplete?: boolean; lastCheckInDate?: string | null }; + result.summary = { + onboardingComplete: state.onboardingComplete ?? false, + lastCheckInDate: state.lastCheckInDate ?? null, + fileCount, + }; + } catch { + result.summary = { + onboardingComplete: false, + lastCheckInDate: null, + fileCount, + }; + } + } + + return NextResponse.json(result); + } catch (e) { + console.error('[workspace/inspect] GET failed:', e); + return NextResponse.json({ error: 'Failed to inspect workspace path' }, { status: 500 }); + } +} diff --git a/src/app/api/workspace/onboarding/route.ts b/src/app/api/workspace/onboarding/route.ts index f16257e..a8b9b69 100644 --- a/src/app/api/workspace/onboarding/route.ts +++ b/src/app/api/workspace/onboarding/route.ts @@ -1,8 +1,8 @@ import { NextRequest, NextResponse } from 'next/server'; import fs from 'fs'; import path from 'path'; -import { getSetting, getDefaultProviderId } from '@/lib/db'; -import { loadState, saveState } from '@/lib/assistant-workspace'; +import { getSetting, getDefaultProviderId, getSession } from '@/lib/db'; +import { loadState, saveState, ensureDailyDir, generateRootDocs } from '@/lib/assistant-workspace'; import { generateTextFromProvider } from '@/lib/text-generator'; const QUESTIONS = [ @@ -16,6 +16,9 @@ const QUESTIONS = [ 'assistant.onboardingQ8', 'assistant.onboardingQ9', 'assistant.onboardingQ10', + 'assistant.onboardingQ11', + 'assistant.onboardingQ12', + 'assistant.onboardingQ13', ]; const QUESTION_LABELS = [ @@ -29,6 +32,9 @@ const QUESTION_LABELS = [ 'What information may be written to long-term memory?', 'What information must never be written to long-term memory?', 'What three things should I do first when entering a project?', + 'How do you organize your materials? (by project / time / topic / mixed)', + 'Where should new information go by default?', + 'How should completed tasks be archived?', ]; export async function GET() { @@ -49,12 +55,20 @@ export async function POST(request: NextRequest) { } const body = await request.json(); - const { answers } = body as { answers: Record }; + const { answers, sessionId } = body as { answers: Record; sessionId?: string }; if (!answers || typeof answers !== 'object') { return NextResponse.json({ error: 'Invalid answers format' }, { status: 400 }); } + // Validate that the calling session belongs to this workspace + if (sessionId) { + const session = getSession(sessionId); + if (session && session.working_directory !== workspacePath) { + return NextResponse.json({ error: 'Session does not belong to current workspace' }, { status: 403 }); + } + } + // Build Q&A text for the prompt const qaText = QUESTION_LABELS.map((q, i) => { const key = `q${i + 1}`; @@ -63,6 +77,8 @@ export async function POST(request: NextRequest) { let soulContent: string; let userContent: string; + let claudeContent: string; + let memoryContent: string; try { const providerId = getDefaultProviderId() || ''; @@ -72,10 +88,22 @@ export async function POST(request: NextRequest) { const userPrompt = `Based on the following user onboarding answers, generate a concise "user.md" profile that captures the user's preferences, goals, and boundaries. Write in third person. Keep it under 2000 characters. Use markdown headers and bullet points.\n\n${qaText}`; - [soulContent, userContent] = await Promise.all([ + const claudePrompt = `Based on the following user onboarding answers, generate a "claude.md" rules file for an AI assistant. Include: +- Execution rules (what to do when entering a project, based on Q10) +- Communication style rules (based on Q3, Q4, Q7) +- Memory rules (what to remember/forget, based on Q8, Q9) +- Hard boundaries (based on Q5) +Keep it under 2000 characters. Use markdown headers and bullet points.\n\n${qaText}`; + + const memoryPrompt = `Based on the following user onboarding answers, generate an initial "memory.md" file with long-term facts about the user worth remembering. Include user goals, preferences, and any stable facts. Keep it under 1000 characters. Use markdown headers.\n\n${qaText}`; + + [soulContent, userContent, claudeContent, memoryContent] = await Promise.all([ generateTextFromProvider({ providerId, model, system: 'You generate configuration files for AI assistants. Output only the file content, no explanations.', prompt: soulPrompt }), generateTextFromProvider({ providerId, model, system: 'You generate user profile documents. Output only the file content, no explanations.', prompt: userPrompt }), + generateTextFromProvider({ providerId, model, system: 'You generate configuration files for AI assistants. Output only the file content, no explanations.', prompt: claudePrompt }), + generateTextFromProvider({ providerId, model, system: 'You generate knowledge files for AI assistants. Output only the file content, no explanations.', prompt: memoryPrompt }), ]); + // Fallback if AI returned empty if (!soulContent.trim() || !userContent.trim()) { throw new Error('AI returned empty content'); @@ -84,18 +112,78 @@ export async function POST(request: NextRequest) { console.warn('[workspace/onboarding] AI generation failed, using raw answers:', e); // Fallback: write raw answers soulContent = `# Soul\n\n## Communication Style\n- Address user as: ${answers.q1 || 'not specified'}\n- Assistant name: ${answers.q2 || 'not specified'}\n- Style: ${answers.q3 || 'not specified'}\n- Approach: ${answers.q4 || 'not specified'}\n`; - userContent = `# User Profile\n\n## Preferences\n- Boundaries: ${answers.q5 || 'not specified'}\n- Goals: ${answers.q6 || 'not specified'}\n- Output format: ${answers.q7 || 'not specified'}\n- Memory allowed: ${answers.q8 || 'not specified'}\n- Memory forbidden: ${answers.q9 || 'not specified'}\n- Project entry: ${answers.q10 || 'not specified'}\n`; + userContent = `# User Profile\n\n## Preferences\n- Boundaries: ${answers.q5 || 'not specified'}\n- Goals: ${answers.q6 || 'not specified'}\n- Output format: ${answers.q7 || 'not specified'}\n- Memory allowed: ${answers.q8 || 'not specified'}\n- Memory forbidden: ${answers.q9 || 'not specified'}\n- Project entry: ${answers.q10 || 'not specified'}\n- Organization: ${answers.q11 || 'not specified'}\n- Default capture: ${answers.q12 || 'not specified'}\n- Archive policy: ${answers.q13 || 'not specified'}\n`; + claudeContent = `# Rules\n\n## Execution\n- On project entry: ${answers.q10 || 'not specified'}\n\n## Boundaries\n- ${answers.q5 || 'not specified'}\n\n## Memory\n- Allowed: ${answers.q8 || 'not specified'}\n- Forbidden: ${answers.q9 || 'not specified'}\n`; + memoryContent = `# Memory\n\n## User Goals\n- ${answers.q6 || 'not specified'}\n`; } - // Write files + // Write all core files fs.writeFileSync(path.join(workspacePath, 'soul.md'), soulContent, 'utf-8'); fs.writeFileSync(path.join(workspacePath, 'user.md'), userContent, 'utf-8'); + if (claudeContent.trim()) { + fs.writeFileSync(path.join(workspacePath, 'claude.md'), claudeContent, 'utf-8'); + } + if (memoryContent.trim()) { + fs.writeFileSync(path.join(workspacePath, 'memory.md'), memoryContent, 'utf-8'); + } + + // Ensure V2 directories + ensureDailyDir(workspacePath); + const inboxDir = path.join(workspacePath, 'Inbox'); + if (!fs.existsSync(inboxDir)) { + fs.mkdirSync(inboxDir, { recursive: true }); + } + + // Generate config.json from answers + try { + const { loadConfig, saveConfig } = await import('@/lib/workspace-config'); + const config = loadConfig(workspacePath); + + // Apply onboarding answers to config + const orgStyle = (answers.q11 || '').toLowerCase(); + if (orgStyle.includes('project')) config.organizationStyle = 'project'; + else if (orgStyle.includes('time')) config.organizationStyle = 'time'; + else if (orgStyle.includes('topic')) config.organizationStyle = 'topic'; + else config.organizationStyle = 'mixed'; + + if (answers.q12) { + // Sanitize captureDefault: only allow relative paths within workspace + let capture = answers.q12.trim(); + if (path.isAbsolute(capture) || capture.startsWith('~') || capture.includes('..')) { + capture = 'Inbox'; // fallback to safe default + } + config.captureDefault = capture; + } + + saveConfig(workspacePath, config); + } catch { + // config module not available, skip + } + + // Generate taxonomy from existing directories + try { + const { loadTaxonomy, saveTaxonomy, inferTaxonomyFromDirs } = await import('@/lib/workspace-taxonomy'); + const taxonomy = loadTaxonomy(workspacePath); + if (taxonomy.categories.length === 0) { + const inferred = inferTaxonomyFromDirs(workspacePath); + if (inferred.length > 0) { + taxonomy.categories = inferred; + saveTaxonomy(workspacePath, taxonomy); + } + } + } catch { + // taxonomy module not available, skip + } + + // Generate root docs + generateRootDocs(workspacePath); // Update state const today = new Date().toISOString().slice(0, 10); const state = loadState(workspacePath); state.onboardingComplete = true; state.lastCheckInDate = today; // Skip daily check-in on the day of onboarding + state.schemaVersion = 2; saveState(workspacePath, state); return NextResponse.json({ success: true }); diff --git a/src/app/api/workspace/organize/route.ts b/src/app/api/workspace/organize/route.ts new file mode 100644 index 0000000..3bdccf8 --- /dev/null +++ b/src/app/api/workspace/organize/route.ts @@ -0,0 +1,98 @@ +import { NextRequest, NextResponse } from 'next/server'; +import path from 'path'; +import { getSetting } from '@/lib/db'; + +interface OrganizeAction { + action: 'capture' | 'classify' | 'move' | 'archive' | 'suggest-evolution'; + title?: string; + content?: string; + filePath?: string; + fromPath?: string; + toPath?: string; +} + +/** Pre-validate that user-supplied paths are safe relative paths within the workspace. */ +function validateRelativePath(p: string, fieldName: string): NextResponse | null { + if (!p) return null; + if (path.isAbsolute(p) || p.startsWith('~') || p.includes('..')) { + return NextResponse.json( + { error: `${fieldName} must be a relative path within the workspace (no absolute paths, ~, or ..)` }, + { status: 400 }, + ); + } + return null; +} + +export async function POST(request: NextRequest) { + try { + const workspacePath = getSetting('assistant_workspace_path'); + if (!workspacePath) { + return NextResponse.json({ error: 'No workspace path configured' }, { status: 400 }); + } + + const body: OrganizeAction = await request.json(); + + // Path safety checks at API boundary + if (body.filePath) { + const err = validateRelativePath(body.filePath, 'filePath'); + if (err) return err; + } + if (body.fromPath) { + const err = validateRelativePath(body.fromPath, 'fromPath'); + if (err) return err; + } + if (body.toPath) { + const err = validateRelativePath(body.toPath, 'toPath'); + if (err) return err; + } + + const organizer = await import('@/lib/workspace-organizer'); + + switch (body.action) { + case 'capture': { + if (!body.title || !body.content) { + return NextResponse.json({ error: 'title and content required' }, { status: 400 }); + } + const filePath = organizer.captureNote(workspacePath, body.title, body.content); + return NextResponse.json({ success: true, filePath }); + } + + case 'classify': { + if (!body.filePath) { + return NextResponse.json({ error: 'filePath required' }, { status: 400 }); + } + const result = organizer.classifyAndSuggest(workspacePath, body.filePath); + return NextResponse.json(result); + } + + case 'move': { + if (!body.fromPath || !body.toPath) { + return NextResponse.json({ error: 'fromPath and toPath required' }, { status: 400 }); + } + organizer.moveFile(workspacePath, body.fromPath, body.toPath); + return NextResponse.json({ success: true }); + } + + case 'archive': { + const result = organizer.archiveDailyMemories(workspacePath); + return NextResponse.json({ success: true, ...result }); + } + + case 'suggest-evolution': { + const suggestions = organizer.suggestTaxonomyEvolution(workspacePath); + return NextResponse.json({ suggestions }); + } + + default: + return NextResponse.json({ error: 'Unknown action' }, { status: 400 }); + } + } catch (e) { + const msg = e instanceof Error ? e.message : 'Organize action failed'; + // Surface containment errors as 400 + if (msg.includes('escapes workspace') || msg.includes('not allowed')) { + return NextResponse.json({ error: msg }, { status: 400 }); + } + console.error('[workspace/organize] POST failed:', e); + return NextResponse.json({ error: 'Organize action failed' }, { status: 500 }); + } +} diff --git a/src/app/api/workspace/search/route.ts b/src/app/api/workspace/search/route.ts new file mode 100644 index 0000000..2b34ebb --- /dev/null +++ b/src/app/api/workspace/search/route.ts @@ -0,0 +1,27 @@ +import { NextRequest, NextResponse } from 'next/server'; +import { getSetting } from '@/lib/db'; + +export async function GET(request: NextRequest) { + try { + const workspacePath = getSetting('assistant_workspace_path'); + if (!workspacePath) { + return NextResponse.json({ error: 'No workspace path configured' }, { status: 400 }); + } + + const { searchParams } = new URL(request.url); + const query = searchParams.get('q'); + const limit = parseInt(searchParams.get('limit') || '5', 10); + + if (!query) { + return NextResponse.json({ error: 'Query parameter "q" is required' }, { status: 400 }); + } + + const { searchWorkspace } = await import('@/lib/workspace-retrieval'); + const results = searchWorkspace(workspacePath, query, { limit }); + + return NextResponse.json({ results }); + } catch (e) { + console.error('[workspace/search] GET failed:', e); + return NextResponse.json({ error: 'Search failed' }, { status: 500 }); + } +} diff --git a/src/app/api/workspace/session/route.ts b/src/app/api/workspace/session/route.ts index 7e06ab0..7bd0911 100644 --- a/src/app/api/workspace/session/route.ts +++ b/src/app/api/workspace/session/route.ts @@ -1,4 +1,5 @@ import { NextRequest, NextResponse } from 'next/server'; +import fs from 'fs'; import { getSetting } from '@/lib/db'; import { getLatestSessionByWorkingDirectory, createSession } from '@/lib/db'; @@ -9,6 +10,17 @@ export async function POST(request: NextRequest) { return NextResponse.json({ error: 'No workspace path configured' }, { status: 400 }); } + // Validate that the workspace directory actually exists and is accessible + try { + const stat = fs.statSync(workspacePath); + if (!stat.isDirectory()) { + return NextResponse.json({ error: 'Workspace path is not a directory', code: 'not_a_directory' }, { status: 400 }); + } + fs.accessSync(workspacePath, fs.constants.R_OK | fs.constants.W_OK); + } catch { + return NextResponse.json({ error: 'Workspace path does not exist or is not accessible', code: 'path_invalid' }, { status: 400 }); + } + const body = await request.json().catch(() => ({})); const { mode } = body as { mode?: 'onboarding' | 'checkin' }; diff --git a/src/components/ai-elements/code-block.tsx b/src/components/ai-elements/code-block.tsx index 9b7bf4e..d92ce8c 100644 --- a/src/components/ai-elements/code-block.tsx +++ b/src/components/ai-elements/code-block.tsx @@ -7,6 +7,7 @@ import type { HighlighterGeneric, ThemedToken, } from "shiki"; +import { bundledLanguages } from "shiki"; import { Button } from "@/components/ui/button"; import { @@ -32,10 +33,10 @@ import { createHighlighter } from "shiki"; // Shiki uses bitflags for font styles: 1=italic, 2=bold, 4=underline // biome-ignore lint/suspicious/noBitwiseOperators: shiki bitflag check -// eslint-disable-next-line no-bitwise -- shiki bitflag check + const isItalic = (fontStyle: number | undefined) => fontStyle && fontStyle & 1; // biome-ignore lint/suspicious/noBitwiseOperators: shiki bitflag check -// eslint-disable-next-line no-bitwise -- shiki bitflag check + // oxlint-disable-next-line eslint(no-bitwise) const isBold = (fontStyle: number | undefined) => fontStyle && fontStyle & 2; const isUnderline = (fontStyle: number | undefined) => @@ -138,24 +139,30 @@ const getTokensCacheKey = (code: string, language: BundledLanguage) => { return `${language}:${code.length}:${start}:${end}`; }; +const isBundledLanguage = (lang: string): lang is BundledLanguage => + lang in bundledLanguages || lang === "text" || lang === "plaintext"; + const getHighlighter = ( language: BundledLanguage ): Promise> => { - const cached = highlighterCache.get(language); + // Normalize unknown languages to "text" before hitting Shiki + const safeLang = isBundledLanguage(language) ? language : ("text" as BundledLanguage); + + const cached = highlighterCache.get(safeLang); if (cached) { return cached; } const highlighterPromise = createHighlighter({ - langs: [language], + langs: [safeLang], themes: ["github-light", "github-dark"], }).catch(() => { // Language not supported by Shiki — fall back to plain text - highlighterCache.delete(language); + highlighterCache.delete(safeLang); return getHighlighter("text" as BundledLanguage); }); - highlighterCache.set(language, highlighterPromise); + highlighterCache.set(safeLang, highlighterPromise); return highlighterPromise; }; diff --git a/src/components/ai-elements/message.tsx b/src/components/ai-elements/message.tsx index 2329b5f..44b1094 100644 --- a/src/components/ai-elements/message.tsx +++ b/src/components/ai-elements/message.tsx @@ -16,7 +16,7 @@ import { } from "@/components/ui/tooltip"; import { cn } from "@/lib/utils"; import { cjk } from "@streamdown/cjk"; -import { code } from "@streamdown/code"; +import { createCodePlugin } from "@streamdown/code"; import { math } from "@streamdown/math"; import { mermaid } from "@streamdown/mermaid"; import { ChevronLeftIcon, ChevronRightIcon } from "lucide-react"; @@ -322,7 +322,18 @@ export const MessageBranchPage = ({ export type MessageResponseProps = ComponentProps; -const streamdownPlugins = { cjk, code, math, mermaid }; +// Wrap @streamdown/code to silently skip unsupported languages instead of throwing +const _codePlugin = createCodePlugin(); +const safeCode: typeof _codePlugin = { + ..._codePlugin, + highlight(params, callback) { + if (!_codePlugin.supportsLanguage(params.language)) { + return null; // Let Streamdown render as plain text + } + return _codePlugin.highlight(params, callback); + }, +}; +const streamdownPlugins = { cjk, code: safeCode, math, mermaid }; export const MessageResponse = memo( ({ className, ...props }: MessageResponseProps) => ( diff --git a/src/components/chat/ChatView.tsx b/src/components/chat/ChatView.tsx index 6b9779e..eda677d 100644 --- a/src/components/chat/ChatView.tsx +++ b/src/components/chat/ChatView.tsx @@ -1,10 +1,12 @@ 'use client'; import { useState, useCallback, useEffect, useRef, useMemo } from 'react'; +import { useRouter } from 'next/navigation'; import type { Message, MessagesResponse, FileAttachment, SessionStreamSnapshot } from '@/types'; import { MessageList } from './MessageList'; import { MessageInput } from './MessageInput'; import { usePanel } from '@/hooks/usePanel'; +import { useTranslation } from '@/hooks/useTranslation'; import { BatchExecutionDashboard, BatchContextSync } from './batch-image-gen'; import { setLastGeneratedImages, transferPendingToMessage } from '@/lib/image-ref-store'; import { @@ -27,7 +29,12 @@ interface ChatViewProps { export function ChatView({ sessionId, initialMessages = [], initialHasMore = false, modelName, initialMode, providerId }: ChatViewProps) { const { setStreamingSessionId, workingDirectory, setWorkingDirectory, setPanelOpen, setPendingApprovalSessionId } = usePanel(); + const { t } = useTranslation(); + const router = useRouter(); const [messages, setMessages] = useState(initialMessages); + + // Workspace mismatch banner state + const [workspaceMismatchPath, setWorkspaceMismatchPath] = useState(null); const [hasMore, setHasMore] = useState(initialHasMore); const [loadingMore, setLoadingMore] = useState(false); const loadingMoreRef = useRef(false); @@ -203,6 +210,11 @@ export function ChatView({ sessionId, initialMessages = [], initialHasMore = fal const state = data.state; if (!state) return; + // Check hookTriggeredSessionId: if this session already has a trigger in progress + // AND there are existing messages (conversation started), skip to avoid re-triggering. + // If the session has no messages, the previous trigger may have failed — allow retry. + if (state.hookTriggeredSessionId === sessionId && initialMessages.length > 0) return; + const today = new Date().toISOString().slice(0, 10); const needsOnboarding = !state.onboardingComplete; const needsCheckIn = state.onboardingComplete && state.lastCheckInDate !== today; @@ -259,6 +271,73 @@ export function ChatView({ sessionId, initialMessages = [], initialHasMore = fal return () => clearTimeout(timer); }, [checkAssistantTrigger]); + // Detect workspace mismatch: only show banner when this session was previously + // an assistant workspace session (has .assistant data) but the path has since changed. + // Normal project chats should never see this banner. + useEffect(() => { + if (!workingDirectory) return; + let cancelled = false; + (async () => { + try { + const res = await fetch('/api/settings/workspace'); + if (!res.ok || cancelled) return; + const data = await res.json(); + if (cancelled) return; + + if (data.path && workingDirectory !== data.path) { + // Only show banner if this session's workingDirectory was itself an assistant workspace + // (i.e., it has .assistant/state.json). Regular project chats skip the banner. + const inspectRes = await fetch(`/api/workspace/inspect?path=${encodeURIComponent(workingDirectory)}`); + if (!inspectRes.ok || cancelled) return; + const inspectData = await inspectRes.json(); + if (inspectData.hasAssistantData) { + setWorkspaceMismatchPath(data.path); + } else { + setWorkspaceMismatchPath(null); + } + } else { + setWorkspaceMismatchPath(null); + } + } catch { + // ignore + } + })(); + return () => { cancelled = true; }; + }, [workingDirectory]); + + // Listen for workspace-switched events (from AssistantWorkspaceSection). + // Only show banner if this session's workingDirectory was the OLD assistant workspace. + useEffect(() => { + const handler = (e: Event) => { + const detail = (e as CustomEvent).detail; + if (detail?.newPath && workingDirectory && workingDirectory === detail.oldPath) { + setWorkspaceMismatchPath(detail.newPath); + } + }; + window.addEventListener('assistant-workspace-switched', handler); + return () => window.removeEventListener('assistant-workspace-switched', handler); + }, [workingDirectory]); + + const handleOpenNewAssistant = useCallback(async () => { + try { + const model = typeof window !== 'undefined' ? localStorage.getItem('codepilot:last-model') || '' : ''; + const provider_id = typeof window !== 'undefined' ? localStorage.getItem('codepilot:last-provider-id') || '' : ''; + // Prefer reusing the latest assistant session (checkin mode) rather than always creating a new onboarding session + const res = await fetch('/api/workspace/session', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ mode: 'checkin', model, provider_id }), + }); + if (res.ok) { + const data = await res.json(); + window.dispatchEvent(new CustomEvent('session-created')); + router.push(`/chat/${data.session.id}`); + } + } catch (e) { + console.error('[ChatView] Failed to open assistant session:', e); + } + }, [router]); + const loadEarlierMessages = useCallback(async () => { // Use ref as atomic lock to prevent double-fetch from rapid clicks if (loadingMoreRef.current || !hasMore || messages.length === 0) return; @@ -296,8 +375,20 @@ export function ChatView({ sessionId, initialMessages = [], initialHasMore = fal [sessionId, setPendingApprovalSessionId] ); - // Detect assistant project completion signals in final message content + // Detect assistant project completion signals in final message content. + // Only processes fences when this session's workingDirectory matches the current assistant workspace path. const detectAssistantCompletion = useCallback(async (content: string) => { + // Guard: only allow fence processing for the current assistant workspace session + if (!workingDirectory) return; + try { + const wsRes = await fetch('/api/settings/workspace'); + if (!wsRes.ok) return; + const wsData = await wsRes.json(); + if (!wsData.path || wsData.path !== workingDirectory) return; + } catch { + return; + } + const clearHookTriggered = () => fetch('/api/workspace/hook-triggered', { method: 'POST', @@ -313,7 +404,7 @@ export function ChatView({ sessionId, initialMessages = [], initialHasMore = fal await fetch('/api/workspace/onboarding', { method: 'POST', headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ answers }), + body: JSON.stringify({ answers, sessionId }), }); await clearHookTriggered(); } catch (e) { @@ -330,14 +421,14 @@ export function ChatView({ sessionId, initialMessages = [], initialHasMore = fal await fetch('/api/workspace/checkin', { method: 'POST', headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ answers }), + body: JSON.stringify({ answers, sessionId }), }); await clearHookTriggered(); } catch (e) { console.error('[ChatView] Check-in completion failed:', e); } } - }, []); + }, [sessionId, workingDirectory]); // Send message — delegates stream management to the manager const sendMessage = useCallback( @@ -508,6 +599,20 @@ export function ChatView({ sessionId, initialMessages = [], initialHasMore = fal return (
+ {/* Workspace mismatch banner */} + {workspaceMismatchPath && ( +
+ + {t('assistant.switchedBanner', { path: workspaceMismatchPath })} + + +
+ )} {workspacePath && group.workingDirectory === workspacePath && ( - + )} {/* New chat in project button (on hover) */} {group.workingDirectory !== "" && ( diff --git a/src/components/settings/AssistantWorkspaceSection.tsx b/src/components/settings/AssistantWorkspaceSection.tsx index 3acced5..4eb76e7 100644 --- a/src/components/settings/AssistantWorkspaceSection.tsx +++ b/src/components/settings/AssistantWorkspaceSection.tsx @@ -1,11 +1,22 @@ "use client"; -import { useState, useCallback, useEffect } from "react"; +import { useState, useCallback, useEffect, useRef } from "react"; import { useRouter } from "next/navigation"; import { Button } from "@/components/ui/button"; +import { + AlertDialog, + AlertDialogAction, + AlertDialogCancel, + AlertDialogContent, + AlertDialogDescription, + AlertDialogFooter, + AlertDialogHeader, + AlertDialogTitle, +} from "@/components/ui/alert-dialog"; import { HugeiconsIcon } from "@hugeicons/react"; -import { Loading02Icon } from "@hugeicons/core-free-icons"; +import { Loading02Icon, CheckmarkCircle02Icon, Cancel01Icon } from "@hugeicons/core-free-icons"; import { useTranslation } from "@/hooks/useTranslation"; +import type { WorkspaceInspectResult } from "@/types"; interface FileStatus { exists: boolean; @@ -19,8 +30,26 @@ interface WorkspaceState { schemaVersion: number; } +interface TaxonomyCategoryInfo { + id: string; + label: string; + role: string; + confidence: number; + source: string; + paths: string[]; +} + +interface IndexStats { + fileCount: number; + chunkCount: number; + lastIndexed: number; + staleCount: number; +} + interface WorkspaceInfo { path: string | null; + valid?: boolean; + reason?: string; exists?: boolean; files: Record; state: WorkspaceState | null; @@ -33,6 +62,17 @@ const FILE_LABELS: Record = { memory: "memory.md", }; +type TabId = 'files' | 'taxonomy' | 'index' | 'organize'; + +type PathValidationStatus = 'idle' | 'checking' | 'valid' | 'invalid'; + +type ConfirmDialogType = + | { kind: 'not_found' } + | { kind: 'empty' } + | { kind: 'normal_directory' } + | { kind: 'existing_workspace'; summary: NonNullable } + | { kind: 'partial_workspace' }; + export function AssistantWorkspaceSection() { const { t } = useTranslation(); const router = useRouter(); @@ -42,6 +82,20 @@ export function AssistantWorkspaceSection() { const [refreshingDocs, setRefreshingDocs] = useState(false); const [pathInput, setPathInput] = useState(""); const [creatingSession, setCreatingSession] = useState(false); + const [activeTab, setActiveTab] = useState('files'); + const [taxonomy, setTaxonomy] = useState([]); + const [indexStats, setIndexStats] = useState(null); + const [reindexing, setReindexing] = useState(false); + const [archiving, setArchiving] = useState(false); + + // Path validation state + const [pathValidation, setPathValidation] = useState('idle'); + const [pathError, setPathError] = useState(null); + const debounceRef = useRef | null>(null); + + // Confirmation dialog state + const [confirmDialog, setConfirmDialog] = useState(null); + const [inspecting, setInspecting] = useState(false); const fetchWorkspace = useCallback(async () => { try { @@ -58,28 +112,211 @@ export function AssistantWorkspaceSection() { } }, []); + const fetchTaxonomy = useCallback(async () => { + try { + const res = await fetch("/api/settings/workspace"); + if (res.ok) { + const data = await res.json(); + if (data.taxonomy) setTaxonomy(data.taxonomy); + } + } catch { /* ignore */ } + }, []); + + const fetchIndexStats = useCallback(async () => { + try { + const res = await fetch("/api/workspace/index"); + if (res.ok) { + const data = await res.json(); + setIndexStats(data); + } + } catch { /* ignore */ } + }, []); + useEffect(() => { fetchWorkspace(); }, [fetchWorkspace]); - const handleSavePath = useCallback(async (initialize: boolean) => { + useEffect(() => { + if (workspace?.path && activeTab === 'taxonomy') fetchTaxonomy(); + if (workspace?.path && activeTab === 'index') fetchIndexStats(); + }, [workspace?.path, activeTab, fetchTaxonomy, fetchIndexStats]); + + // Debounced path validation + const validatePath = useCallback((path: string) => { + if (debounceRef.current) clearTimeout(debounceRef.current); + setPathError(null); + + if (!path.trim()) { + setPathValidation('idle'); + return; + } + + setPathValidation('checking'); + debounceRef.current = setTimeout(async () => { + try { + const res = await fetch(`/api/workspace/inspect?path=${encodeURIComponent(path.trim())}`); + if (!res.ok) { + setPathValidation('invalid'); + setPathError(t('assistant.inspectFailed')); + return; + } + const data: WorkspaceInspectResult = await res.json(); + if (!data.exists) { + // Non-existent path is allowed — user can create it via initialize + setPathValidation('valid'); + } else if (!data.isDirectory) { + setPathValidation('invalid'); + setPathError(t('assistant.pathNotDirectory')); + } else if (!data.readable) { + setPathValidation('invalid'); + setPathError(t('assistant.pathNotReadable')); + } else if (!data.writable) { + setPathValidation('invalid'); + setPathError(t('assistant.pathNotWritable')); + } else { + setPathValidation('valid'); + } + } catch { + setPathValidation('invalid'); + setPathError(t('assistant.inspectFailed')); + } + }, 500); + }, [t]); + + // Clean up debounce on unmount + useEffect(() => { + return () => { + if (debounceRef.current) clearTimeout(debounceRef.current); + }; + }, []); + + const handlePathInputChange = useCallback((value: string) => { + setPathInput(value); + validatePath(value); + }, [validatePath]); + + // Execute the actual save + optional auto-navigate + // navigateMode: 'new' = always create new session, 'reuse' = try to reuse latest session + const executeSave = useCallback(async (initialize: boolean, resetOnboarding?: boolean, navigateMode: 'new' | 'reuse' = 'new') => { if (!pathInput.trim()) return; + const oldPath = workspace?.path || null; + const newPath = pathInput.trim(); if (initialize) setInitializing(true); try { + const body: Record = { path: newPath, initialize }; + if (resetOnboarding) body.resetOnboarding = true; const res = await fetch("/api/settings/workspace", { method: "PUT", headers: { "Content-Type": "application/json" }, - body: JSON.stringify({ path: pathInput.trim(), initialize }), + body: JSON.stringify(body), }); if (res.ok) { await fetchWorkspace(); + + // If path actually changed, dispatch event + if (oldPath && oldPath !== newPath) { + window.dispatchEvent(new CustomEvent('assistant-workspace-switched', { + detail: { oldPath, newPath }, + })); + } + + // Navigate to assistant session + try { + const model = typeof window !== 'undefined' ? localStorage.getItem('codepilot:last-model') || '' : ''; + const provider_id = typeof window !== 'undefined' ? localStorage.getItem('codepilot:last-provider-id') || '' : ''; + const sessionMode = navigateMode === 'reuse' ? 'checkin' : 'onboarding'; + const sessionRes = await fetch("/api/workspace/session", { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ mode: sessionMode, model, provider_id }), + }); + if (sessionRes.ok) { + const sessionData = await sessionRes.json(); + window.dispatchEvent(new CustomEvent("session-created")); + router.push(`/chat/${sessionData.session.id}`); + } + } catch (navErr) { + console.error("Failed to auto-navigate to session:", navErr); + } } } catch (e) { console.error("Failed to save workspace path:", e); } finally { setInitializing(false); + setConfirmDialog(null); } - }, [pathInput, fetchWorkspace]); + }, [pathInput, fetchWorkspace, workspace?.path, router]); + + // Inspect path and show confirmation dialog (or save directly if path unchanged) + const handleSaveClick = useCallback(async () => { + if (!pathInput.trim()) return; + + // If path hasn't changed, no-op + if (pathInput.trim() === workspace?.path) { + return; + } + + setInspecting(true); + try { + const res = await fetch(`/api/workspace/inspect?path=${encodeURIComponent(pathInput.trim())}`); + if (!res.ok) { + setPathValidation('invalid'); + setPathError(t('assistant.inspectFailed')); + return; + } + const data: WorkspaceInspectResult = await res.json(); + + // Non-existent path — offer to create it + if (!data.exists) { + setConfirmDialog({ kind: 'not_found' }); + return; + } + // Check for invalid states — show error, block save + if (!data.isDirectory) { + setPathValidation('invalid'); + setPathError(t('assistant.pathNotDirectory')); + return; + } + if (!data.readable) { + setPathValidation('invalid'); + setPathError(t('assistant.pathNotReadable')); + return; + } + if (!data.writable) { + setPathValidation('invalid'); + setPathError(t('assistant.pathNotWritable')); + return; + } + + // Show confirmation dialog based on workspace status + switch (data.workspaceStatus) { + case 'empty': + setConfirmDialog({ kind: 'empty' }); + break; + case 'normal_directory': + setConfirmDialog({ kind: 'normal_directory' }); + break; + case 'existing_workspace': + setConfirmDialog({ + kind: 'existing_workspace', + summary: data.summary || { onboardingComplete: false, lastCheckInDate: null, fileCount: 0 }, + }); + break; + case 'partial_workspace': + setConfirmDialog({ kind: 'partial_workspace' }); + break; + default: + setPathValidation('invalid'); + setPathError(t('assistant.pathInvalid')); + } + } catch (e) { + console.error("Failed to inspect workspace:", e); + setPathValidation('invalid'); + setPathError(t('assistant.inspectFailed')); + } finally { + setInspecting(false); + } + }, [pathInput, workspace?.path, executeSave, t]); const handleSelectFolder = useCallback(async () => { try { @@ -88,15 +325,19 @@ export function AssistantWorkspaceSection() { const selected = await w.electronAPI.selectFolder(); if (selected) { setPathInput(selected); + validatePath(selected); } } else { const input = prompt("Enter workspace directory path:"); - if (input) setPathInput(input); + if (input) { + setPathInput(input); + validatePath(input); + } } } catch (e) { console.error("Failed to select folder:", e); } - }, []); + }, [validatePath]); const handleRefreshDocs = useCallback(async () => { setRefreshingDocs(true); @@ -109,7 +350,6 @@ export function AssistantWorkspaceSection() { } }, []); - /** Create a new onboarding session in the assistant project folder and navigate to it */ const handleStartOnboarding = useCallback(async () => { if (!workspace?.path) return; setCreatingSession(true); @@ -133,7 +373,6 @@ export function AssistantWorkspaceSection() { } }, [workspace?.path, router]); - /** Create or reuse a check-in session in the assistant project folder and navigate to it */ const handleStartCheckIn = useCallback(async () => { if (!workspace?.path) return; setCreatingSession(true); @@ -157,6 +396,33 @@ export function AssistantWorkspaceSection() { } }, [workspace?.path, router]); + const handleReindex = useCallback(async () => { + setReindexing(true); + try { + await fetch("/api/workspace/index", { method: "POST" }); + await fetchIndexStats(); + } catch (e) { + console.error("Failed to reindex:", e); + } finally { + setReindexing(false); + } + }, [fetchIndexStats]); + + const handleArchive = useCallback(async () => { + setArchiving(true); + try { + await fetch("/api/workspace/organize", { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ action: 'archive' }), + }); + } catch (e) { + console.error("Failed to archive:", e); + } finally { + setArchiving(false); + } + }, []); + if (loading) { return (
@@ -168,6 +434,27 @@ export function AssistantWorkspaceSection() { const today = new Date().toISOString().slice(0, 10); const checkInDoneToday = workspace?.state?.lastCheckInDate === today; + const tabs: Array<{ id: TabId; label: string }> = [ + { id: 'files', label: t('assistant.fileStatus') }, + { id: 'taxonomy', label: t('assistant.taxonomyTitle') }, + { id: 'index', label: t('assistant.indexTitle') }, + { id: 'organize', label: t('assistant.organizeTitle') }, + ]; + + // Render path validation indicator + const renderValidationIcon = () => { + switch (pathValidation) { + case 'checking': + return ; + case 'valid': + return ; + case 'invalid': + return ; + default: + return null; + } + }; + return (
{/* Workspace Path Card */} @@ -175,74 +462,63 @@ export function AssistantWorkspaceSection() {

{t('assistant.workspacePath')}

{t('assistant.workspacePathHint')}

- setPathInput(e.target.value)} - placeholder="/path/to/workspace" - className="flex-1 rounded-md border border-border/50 bg-background px-3 py-1.5 text-sm" - /> +
+ handlePathInputChange(e.target.value)} + placeholder="/path/to/workspace" + className="w-full rounded-md border border-border/50 bg-background px-3 py-1.5 text-sm pr-8" + /> +
+ {renderValidationIcon()} +
+
+ {pathError && ( +

{pathError}

+ )}
-
- {/* File Status Card */} - {workspace?.path && ( -
-

{t('assistant.fileStatus')}

-
- {Object.entries(FILE_LABELS).map(([key, label]) => { - const file = workspace.files[key]; - return ( -
- {label} -
- {file?.exists ? ( - <> - - {t('assistant.fileChars', { count: String(file.chars) })} - - - {t('assistant.fileExists')} - - ) : ( - <> - - {t('assistant.fileMissing')} - - )} -
-
- ); - })} -
+ {/* Invalid workspace path warning */} + {workspace?.path && workspace.valid === false && ( +
+

+ {t('assistant.workspaceInvalid')}: {workspace.reason === 'path_not_found' + ? t('assistant.pathNotExist') + : workspace.reason === 'not_a_directory' + ? t('assistant.pathNotDirectory') + : workspace.reason === 'not_readable' + ? t('assistant.pathNotReadable') + : workspace.reason === 'not_writable' + ? t('assistant.pathNotWritable') + : t('assistant.pathInvalid') + } +

)} {/* Onboarding Status Card */} - {workspace?.path && ( + {workspace?.path && workspace.valid !== false && (
@@ -272,8 +548,8 @@ export function AssistantWorkspaceSection() {
)} - {/* Daily Check-in Card */} - {workspace?.path && ( + {/* Daily Check-in Card — only shown when onboarding is complete */} + {workspace?.path && workspace.valid !== false && workspace.state?.onboardingComplete && (
@@ -308,31 +584,326 @@ export function AssistantWorkspaceSection() {
)} - {/* Directory Docs Card */} - {workspace?.path && ( + {/* Tabbed Section: Files / Taxonomy / Index / Organize */} + {workspace?.path && workspace.valid !== false && (
-
-
-

{t('assistant.refreshDocs')}

-
- +
+ {tabs.map(tab => ( + + ))}
+ + {/* Files Tab */} + {activeTab === 'files' && ( +
+ {Object.entries(FILE_LABELS).map(([key, label]) => { + const file = workspace.files[key]; + return ( +
+ {label} +
+ {file?.exists ? ( + <> + + {t('assistant.fileChars', { count: String(file.chars) })} + + + {t('assistant.fileExists')} + + ) : ( + <> + + {t('assistant.fileMissing')} + + )} +
+
+ ); + })} +
+ +
+
+ )} + + {/* Taxonomy Tab */} + {activeTab === 'taxonomy' && ( +
+

{t('assistant.taxonomyDesc')}

+ {taxonomy.length === 0 ? ( +

{t('assistant.taxonomyEmpty')}

+ ) : ( +
+ {taxonomy.map(cat => ( +
+
+ {cat.label} + {t('assistant.taxonomyRole')}: {cat.role} +
+
+ {t('assistant.taxonomySource')}: {cat.source} + 0.7 ? 'bg-green-500/10 text-green-600' : + cat.confidence > 0.4 ? 'bg-yellow-500/10 text-yellow-600' : + 'bg-red-500/10 text-red-600' + }`}> + {Math.round(cat.confidence * 100)}% + +
+
+ ))} +
+ )} +
+ )} + + {/* Index Tab */} + {activeTab === 'index' && ( +
+

{t('assistant.indexDesc')}

+ {indexStats ? ( +
+
+ {t('assistant.indexFiles', { count: String(indexStats.fileCount) })} +
+
+ {t('assistant.indexChunks', { count: String(indexStats.chunkCount) })} +
+
+ {t('assistant.indexStale', { count: String(indexStats.staleCount) })} +
+
+ + {t('assistant.indexLastIndexed')}: {indexStats.lastIndexed ? new Date(indexStats.lastIndexed).toLocaleString() : 'never'} + +
+
+ ) : ( +

{t('common.loading')}

+ )} +
+ +
+
+ )} + + {/* Organize Tab */} + {activeTab === 'organize' && ( +
+

{t('assistant.organizeDesc')}

+
+ +
+
+ )}
)} + + {/* ── Confirmation Dialogs ── */} + + {/* Non-existent path — offer to create */} + { if (!open) setConfirmDialog(null); }}> + + + {t('assistant.confirmNotFoundTitle')} + + {t('assistant.confirmNotFoundDesc')} + + + + {t('common.cancel')} + executeSave(true)} disabled={initializing}> + {initializing ? ( + <> + + {t('assistant.initializing')} + + ) : ( + t('assistant.confirmCreate') + )} + + + + + + {/* Empty directory confirmation */} + { if (!open) setConfirmDialog(null); }}> + + + {t('assistant.confirmEmptyTitle')} + + {t('assistant.confirmEmptyDesc')} + + + + {t('common.cancel')} + executeSave(true)} disabled={initializing}> + {initializing ? ( + <> + + {t('assistant.initializing')} + + ) : ( + t('assistant.confirmInitialize') + )} + + + + + + {/* Normal directory confirmation */} + { if (!open) setConfirmDialog(null); }}> + + + {t('assistant.confirmNormalTitle')} + +
+

{t('assistant.confirmNormalDesc')}

+

{t('assistant.confirmNormalHint')}

+
+ + + + {t('common.cancel')} + executeSave(true)} disabled={initializing}> + {initializing ? ( + <> + + {t('assistant.initializing')} + + ) : ( + t('assistant.confirmInitialize') + )} + + + + + + {/* Existing workspace confirmation */} + { if (!open) setConfirmDialog(null); }}> + + + {t('assistant.confirmExistingTitle')} + +
+

{t('assistant.confirmExistingDesc')}

+ {confirmDialog?.kind === 'existing_workspace' && ( +
+
+ {t('assistant.summaryOnboarding')}: + {confirmDialog.summary.onboardingComplete + ? t('assistant.onboardingComplete') + : t('assistant.onboardingNotStarted') + } +
+
+ {t('assistant.summaryLastCheckIn')}: + {confirmDialog.summary.lastCheckInDate || t('assistant.summaryNever')} +
+
+ {t('assistant.summaryFileCount')}: + {confirmDialog.summary.fileCount} +
+
+ )} +
+ + + + + + {t('common.cancel')} + + + + + {/* Partial workspace confirmation */} + { if (!open) setConfirmDialog(null); }}> + + + {t('assistant.confirmPartialTitle')} + + {t('assistant.confirmPartialDesc')} + + + + {t('common.cancel')} + executeSave(true)} disabled={initializing}> + {initializing ? ( + <> + + {t('assistant.initializing')} + + ) : ( + t('assistant.confirmRepair') + )} + + + +
); } diff --git a/src/i18n/en.ts b/src/i18n/en.ts index 77c3547..0413ffc 100644 --- a/src/i18n/en.ts +++ b/src/i18n/en.ts @@ -656,6 +656,9 @@ const en = { 'assistant.onboardingQ8': 'What information may be written to long-term memory?', 'assistant.onboardingQ9': 'What information must never be written to long-term memory?', 'assistant.onboardingQ10': 'What three things should I do first when entering a project?', + 'assistant.onboardingQ11': 'How do you organize your materials? (by project / time / topic / mixed)', + 'assistant.onboardingQ12': 'Where should new information go by default?', + 'assistant.onboardingQ13': 'How should completed tasks be archived?', 'assistant.checkInTitle': 'Daily Check-in', 'assistant.checkInDesc': 'Quick daily update to keep your assistant current', 'assistant.lastCheckIn': 'Last check-in', @@ -668,6 +671,76 @@ const en = { 'assistant.refreshDocs': 'Refresh Directory Docs', 'assistant.refreshingDocs': 'Refreshing...', 'assistant.docsRefreshed': 'Directory docs refreshed', + + // ── Taxonomy ────────────────────────────────────────── + 'assistant.taxonomyTitle': 'Taxonomy', + 'assistant.taxonomyDesc': 'Categories learned from your workspace structure', + 'assistant.taxonomyEmpty': 'No categories detected', + 'assistant.taxonomyRole': 'Role', + 'assistant.taxonomyConfidence': 'Confidence', + 'assistant.taxonomySource': 'Source', + 'assistant.taxonomyPaths': 'Paths', + + // ── Index ───────────────────────────────────────────── + 'assistant.indexTitle': 'File Index', + 'assistant.indexDesc': 'Indexed files for search and retrieval', + 'assistant.indexFiles': '{count} files indexed', + 'assistant.indexChunks': '{count} chunks', + 'assistant.indexStale': '{count} stale', + 'assistant.indexLastIndexed': 'Last indexed', + 'assistant.indexReindex': 'Reindex', + 'assistant.indexReindexing': 'Reindexing...', + + // ── Hotset ──────────────────────────────────────────── + 'assistant.hotsetTitle': 'Hotset', + 'assistant.hotsetDesc': 'Frequently accessed files loaded into context', + 'assistant.hotsetEmpty': 'No frequent files yet', + 'assistant.hotsetPinned': 'Pinned', + 'assistant.hotsetFrequent': 'Frequent', + + // ── Organize ────────────────────────────────────────── + 'assistant.organizeTitle': 'Organization', + 'assistant.organizeDesc': 'Archive and organize workspace content', + 'assistant.organizeArchive': 'Archive Old Memories', + 'assistant.organizeArchiving': 'Archiving...', + 'assistant.organizeArchived': '{count} memories archived', + 'assistant.organizeSuggest': 'Suggest Evolution', + 'assistant.organizeSuggesting': 'Analyzing...', + + // ── Assistant Workspace Switch Banner ───────────────── + 'assistant.switchedBanner': 'Assistant directory has been switched to {path}. This session is no longer bound to assistant features.', + 'assistant.openNewAssistant': 'Open New Assistant Project', + 'assistant.sessionUnbound': 'Session unbound from assistant', + + // ── Workspace Inspect & Confirmation ────────────────── + 'assistant.inspecting': 'Checking...', + 'assistant.inspectFailed': 'Failed to inspect path', + 'assistant.pathNotExist': 'Path does not exist', + 'assistant.pathNotDirectory': 'Path is not a directory', + 'assistant.pathNotReadable': 'Directory is not readable', + 'assistant.pathNotWritable': 'Directory is not writable', + 'assistant.pathInvalid': 'Invalid path', + 'assistant.confirmEmptyTitle': 'Initialize New Workspace', + 'assistant.confirmEmptyDesc': 'This is an empty directory. Initialize it as a new assistant workspace?', + 'assistant.confirmNormalTitle': 'Initialize Workspace Here', + 'assistant.confirmNormalDesc': 'This directory already contains files. Initialize assistant workspace here?', + 'assistant.confirmNormalHint': 'An .assistant folder, soul.md, memory.md and other assistant files will be created.', + 'assistant.confirmInitialize': 'Initialize', + 'assistant.confirmExistingTitle': 'Existing Workspace Detected', + 'assistant.confirmExistingDesc': 'This directory already contains an assistant workspace. How would you like to proceed?', + 'assistant.summaryOnboarding': 'Onboarding', + 'assistant.summaryLastCheckIn': 'Last Check-in', + 'assistant.summaryFileCount': 'Files', + 'assistant.summaryNever': 'Never', + 'assistant.takeoverContinue': 'Continue with existing config', + 'assistant.takeoverReonboard': 'Re-onboard (keep files, reset onboarding)', + 'assistant.confirmPartialTitle': 'Partial Workspace Detected', + 'assistant.confirmPartialDesc': 'Partial assistant data was found in this directory. Repair and continue using it?', + 'assistant.confirmRepair': 'Repair & Continue', + 'assistant.confirmNotFoundTitle': 'Create New Directory', + 'assistant.confirmNotFoundDesc': 'This path does not exist yet. Create the directory and initialize it as a new assistant workspace?', + 'assistant.confirmCreate': 'Create & Initialize', + 'assistant.workspaceInvalid': 'Configured workspace path is invalid', } as const; export type TranslationKey = keyof typeof en; diff --git a/src/i18n/zh.ts b/src/i18n/zh.ts index 2f7b977..4683381 100644 --- a/src/i18n/zh.ts +++ b/src/i18n/zh.ts @@ -653,6 +653,9 @@ const zh: Record = { 'assistant.onboardingQ8': '哪些信息允许写入长期记忆？', 'assistant.onboardingQ9': '哪些信息绝对不要写入长期记忆？', 'assistant.onboardingQ10': '每次进入项目我应先做哪三件事？', + 'assistant.onboardingQ11': '你如何组织材料？（按项目 / 时间 / 主题 / 混合）', + 'assistant.onboardingQ12': '新信息默认存放在哪里？', + 'assistant.onboardingQ13': '已完成的任务应如何归档？', 'assistant.checkInTitle': '每日问询', 'assistant.checkInDesc': '每日快速更新以保持助理信息同步', 'assistant.lastCheckIn': '上次问询', @@ -665,6 +668,76 @@ const zh: Record = { 'assistant.refreshDocs': '刷新目录文档', 'assistant.refreshingDocs': '刷新中...', 'assistant.docsRefreshed': '目录文档已刷新', + + // ── Taxonomy ────────────────────────────────────────── + 'assistant.taxonomyTitle': '分类体系', + 'assistant.taxonomyDesc': '从工作区结构学习的分类', + 'assistant.taxonomyEmpty': '未检测到分类', + 'assistant.taxonomyRole': '角色', + 'assistant.taxonomyConfidence': '置信度', + 'assistant.taxonomySource': '来源', + 'assistant.taxonomyPaths': '路径', + + // ── Index ───────────────────────────────────────────── + 'assistant.indexTitle': '文件索引', + 'assistant.indexDesc': '已索引的文件用于搜索和检索', + 'assistant.indexFiles': '已索引 {count} 个文件', + 'assistant.indexChunks': '{count} 个分块', + 'assistant.indexStale': '{count} 个过期', + 'assistant.indexLastIndexed': '上次索引', + 'assistant.indexReindex': '重新索引', + 'assistant.indexReindexing': '索引中...', + + // ── Hotset ──────────────────────────────────────────── + 'assistant.hotsetTitle': '热点文件', + 'assistant.hotsetDesc': '频繁访问的文件会加载到上下文中', + 'assistant.hotsetEmpty': '暂无频繁访问的文件', + 'assistant.hotsetPinned': '已固定', + 'assistant.hotsetFrequent': '高频访问', + + // ── Organize ────────────────────────────────────────── + 'assistant.organizeTitle': '组织管理', + 'assistant.organizeDesc': '归档和整理工作区内容', + 'assistant.organizeArchive': '归档旧记忆', + 'assistant.organizeArchiving': '归档中...', + 'assistant.organizeArchived': '已归档 {count} 条记忆', + 'assistant.organizeSuggest': '建议优化', + 'assistant.organizeSuggesting': '分析中...', + + // ── Assistant Workspace Switch Banner ───────────────── + 'assistant.switchedBanner': '助理目录已切换到 {path}。当前会话不再绑定助理功能。', + 'assistant.openNewAssistant': '打开新助理项目', + 'assistant.sessionUnbound': '会话已与助理解绑', + + // ── Workspace Inspect & Confirmation ────────────────── + 'assistant.inspecting': '检查中...', + 'assistant.inspectFailed': '检查路径失败', + 'assistant.pathNotExist': '路径不存在', + 'assistant.pathNotDirectory': '路径不是目录', + 'assistant.pathNotReadable': '目录不可读', + 'assistant.pathNotWritable': '目录不可写', + 'assistant.pathInvalid': '无效路径', + 'assistant.confirmEmptyTitle': '初始化新工作区', + 'assistant.confirmEmptyDesc': '这是一个空目录，是否将其初始化为新的助理工作区？', + 'assistant.confirmNormalTitle': '在此初始化工作区', + 'assistant.confirmNormalDesc': '此目录中已有文件，是否在此初始化助理工作区？', + 'assistant.confirmNormalHint': '将创建 .assistant 文件夹、soul.md、memory.md 等助理文件。', + 'assistant.confirmInitialize': '初始化', + 'assistant.confirmExistingTitle': '检测到已有工作区', + 'assistant.confirmExistingDesc': '此目录已包含助理工作区，您希望如何处理？', + 'assistant.summaryOnboarding': '引导状态', + 'assistant.summaryLastCheckIn': '上次问询', + 'assistant.summaryFileCount': '文件数', + 'assistant.summaryNever': '从未', + 'assistant.takeoverContinue': '继续使用已有配置', + 'assistant.takeoverReonboard': '重新引导（保留文件，重置引导状态）', + 'assistant.confirmPartialTitle': '检测到不完整工作区', + 'assistant.confirmPartialDesc': '在此目录中发现了部分助理数据，是否修复并继续使用？', + 'assistant.confirmRepair': '修复并继续', + 'assistant.confirmNotFoundTitle': '创建新目录', + 'assistant.confirmNotFoundDesc': '此路径尚不存在，是否创建目录并初始化为新的助理工作区？', + 'assistant.confirmCreate': '创建并初始化', + 'assistant.workspaceInvalid': '已配置的工作区路径无效', }; export default zh; diff --git a/src/lib/assistant-workspace.ts b/src/lib/assistant-workspace.ts index d676d2e..d31dcdc 100644 --- a/src/lib/assistant-workspace.ts +++ b/src/lib/assistant-workspace.ts @@ -1,15 +1,16 @@ import fs from 'fs'; import path from 'path'; -import type { AssistantWorkspaceState, AssistantWorkspaceFiles } from '@/types'; +import type { AssistantWorkspaceState, AssistantWorkspaceFiles, AssistantWorkspaceFilesV2, SearchResult } from '@/types'; const DEFAULT_STATE: AssistantWorkspaceState = { onboardingComplete: false, lastCheckInDate: null, - schemaVersion: 1, + schemaVersion: 2, }; const STATE_DIR = '.assistant'; const STATE_FILE = 'state.json'; +const MEMORY_DAILY_DIR = 'memory/daily'; // Canonical filenames — lowercase preferred, uppercase fallback const FILE_MAP: Record = { @@ -31,6 +32,10 @@ const PER_FILE_LIMIT = 8000; const HEAD_SIZE = 6000; const TAIL_SIZE = 1800; const TOTAL_PROMPT_LIMIT = 40000; +const DAILY_MEMORY_LIMIT = 4000; +const ROOT_DOC_LIMIT = 2000; +const RETRIEVAL_RESULT_LIMIT = 3000; +const MAX_RETRIEVAL_RESULTS = 5; function resolveFile(dir: string, key: keyof AssistantWorkspaceFiles): { filePath: string; exists: boolean } { for (const variant of FILE_MAP[key]) { @@ -39,10 +44,167 @@ function resolveFile(dir: string, key: keyof AssistantWorkspaceFiles): { filePat return { filePath, exists: true }; } } - // Return canonical (first/lowercase) path when none exist return { filePath: path.join(dir, FILE_MAP[key][0]), exists: false }; } +// ========================================== +// Daily Memory +// ========================================== + +export function ensureDailyDir(dir: string): string { + const dailyDir = path.join(dir, MEMORY_DAILY_DIR); + if (!fs.existsSync(dailyDir)) { + fs.mkdirSync(dailyDir, { recursive: true }); + } + return dailyDir; +} + +export function writeDailyMemory(dir: string, date: string, content: string): string { + const dailyDir = ensureDailyDir(dir); + const filePath = path.join(dailyDir, `${date}.md`); + fs.writeFileSync(filePath, content, 'utf-8'); + return filePath; +} + +export function loadDailyMemories(dir: string, count = 2): Array<{ date: string; content: string }> { + const dailyDir = path.join(dir, MEMORY_DAILY_DIR); + if (!fs.existsSync(dailyDir)) return []; + + try { + const files = fs.readdirSync(dailyDir) + .filter(f => /^\d{4}-\d{2}-\d{2}\.md$/.test(f)) + .sort() + .reverse() + .slice(0, count); + + return files.map(f => ({ + date: f.replace('.md', ''), + content: fs.readFileSync(path.join(dailyDir, f), 'utf-8'), + })); + } catch { + return []; + } +} + +// ========================================== +// State Migration +// ========================================== + +export function migrateStateV1ToV2(dir: string): void { + // Read state directly (not via loadState, which triggers migration recursively) + let state: AssistantWorkspaceState; + try { + const statePath = path.join(dir, STATE_DIR, STATE_FILE); + const raw = fs.readFileSync(statePath, 'utf-8'); + state = JSON.parse(raw) as AssistantWorkspaceState; + } catch { + return; // No state file, nothing to migrate + } + + if (state.schemaVersion >= 2) return; + + // Create daily memory directory + ensureDailyDir(dir); + + // Create Inbox if not exists + const inboxDir = path.join(dir, 'Inbox'); + if (!fs.existsSync(inboxDir)) { + fs.mkdirSync(inboxDir, { recursive: true }); + } + + // Update schema version + state.schemaVersion = 2; + saveState(dir, state); +} + +// ========================================== +// Root Docs +// ========================================== + +export function generateRootDocs(dir: string): string[] { + const generated: string[] = []; + + // Scan top-level entries + let entries: fs.Dirent[]; + try { + entries = fs.readdirSync(dir, { withFileTypes: true }); + } catch { + return generated; + } + + const subdirs = entries.filter(e => e.isDirectory() && !e.name.startsWith('.')); + const files = entries.filter(e => e.isFile()); + + const startMarker = ''; + const endMarker = ''; + + // README.ai.md — workspace purpose + categories + structure + const dirList = subdirs.map(d => `- **${d.name}/** — directory`).join('\n'); + const fileList = files + .filter(f => !f.name.startsWith('.')) + .map(f => `- ${f.name}`) + .join('\n'); + + const readmeContent = `${startMarker} +# Workspace Overview + +## Directories +${dirList || '(none)'} + +## Root Files +${fileList || '(none)'} + +## Structure +This workspace contains ${subdirs.length} directories and ${files.length} root files. +${endMarker}`; + + const readmePath = path.join(dir, 'README.ai.md'); + writeAiDoc(readmePath, readmeContent, startMarker, endMarker); + generated.push(readmePath); + + // PATH.ai.md — directory tree + role map + const treeLines = [ + `${startMarker}`, + `# Workspace Path Index`, + '', + `Base: \`${dir}\``, + '', + ]; + for (const d of subdirs) { + treeLines.push(`- \`${d.name}/\``); + } + for (const f of files.filter(f => !f.name.startsWith('.'))) { + treeLines.push(`- \`${f.name}\``); + } + treeLines.push(endMarker); + + const pathFilePath = path.join(dir, 'PATH.ai.md'); + writeAiDoc(pathFilePath, treeLines.join('\n'), startMarker, endMarker); + generated.push(pathFilePath); + + return generated; +} + +function writeAiDoc(filePath: string, content: string, startMarker: string, endMarker: string): void { + if (fs.existsSync(filePath)) { + const existing = fs.readFileSync(filePath, 'utf-8'); + const startIdx = existing.indexOf(startMarker); + const endIdx = existing.indexOf(endMarker); + if (startIdx !== -1 && endIdx !== -1) { + const updated = existing.slice(0, startIdx) + content + existing.slice(endIdx + endMarker.length); + fs.writeFileSync(filePath, updated, 'utf-8'); + } else { + fs.writeFileSync(filePath, content + '\n', 'utf-8'); + } + } else { + fs.writeFileSync(filePath, content + '\n', 'utf-8'); + } +} + +// ========================================== +// Validation & Initialization +// ========================================== + export function validateWorkspace(dir: string): { exists: boolean; files: Record; @@ -75,8 +237,18 @@ export function initializeWorkspace(dir: string): string[] { } const created: string[] = []; - const keys = Object.keys(FILE_MAP) as Array; + // Detect if this is an existing directory with content + let existingEntries: fs.Dirent[] = []; + try { + existingEntries = fs.readdirSync(dir, { withFileTypes: true }); + } catch { /* empty */ } + const hasExistingContent = existingEntries.some( + e => !e.name.startsWith('.') && e.name !== 'Inbox' && e.name !== 'memory' + ); + + // Create core workspace files (only if missing) + const keys = Object.keys(FILE_MAP) as Array; for (const key of keys) { const resolved = resolveFile(dir, key); if (!resolved.exists) { @@ -86,21 +258,57 @@ export function initializeWorkspace(dir: string): string[] { } } + // Create V2 directories + ensureDailyDir(dir); + const inboxDir = path.join(dir, 'Inbox'); + if (!fs.existsSync(inboxDir)) { + fs.mkdirSync(inboxDir, { recursive: true }); + } + + // State file const statePath = path.join(stateDir, STATE_FILE); if (!fs.existsSync(statePath)) { saveState(dir, { ...DEFAULT_STATE }); + } else { + // Migrate existing v1 state + migrateStateV1ToV2(dir); + } + + // For existing directories, generate root docs and infer taxonomy + if (hasExistingContent) { + generateRootDocs(dir); + try { + // eslint-disable-next-line @typescript-eslint/no-require-imports + const { inferTaxonomyFromDirs, loadTaxonomy, saveTaxonomy } = require('@/lib/workspace-taxonomy'); + const taxonomy = loadTaxonomy(dir); + if (taxonomy.categories.length === 0) { + const inferred = inferTaxonomyFromDirs(dir); + if (inferred.length > 0) { + taxonomy.categories = inferred; + saveTaxonomy(dir, taxonomy); + } + } + } catch { + // taxonomy module not available, skip + } } return created; } +// ========================================== +// File Loading +// ========================================== + export function truncateContent(content: string, limit: number): string { if (content.length <= limit) return content; - return content.slice(0, HEAD_SIZE) + '\n\n[...truncated...]\n\n' + content.slice(-TAIL_SIZE); + const headSize = Math.min(HEAD_SIZE, Math.floor(limit * 0.75)); + const tailSize = Math.min(TAIL_SIZE, limit - headSize - 30); + return content.slice(0, headSize) + '\n\n[...truncated...]\n\n' + content.slice(-tailSize); } -export function loadWorkspaceFiles(dir: string): AssistantWorkspaceFiles { - const result: AssistantWorkspaceFiles = {}; +export function loadWorkspaceFiles(dir: string): AssistantWorkspaceFilesV2 { + const result: AssistantWorkspaceFilesV2 = {}; const keys = Object.keys(FILE_MAP) as Array; for (const key of keys) { @@ -111,36 +319,151 @@ export function loadWorkspaceFiles(dir: string): AssistantWorkspaceFiles { } } + // Load daily memories (today + yesterday) + const dailyMemories = loadDailyMemories(dir, 2); + if (dailyMemories.length > 0) { + result.dailyMemories = dailyMemories.map(m => + truncateContent(`## ${m.date}\n${m.content}`, DAILY_MEMORY_LIMIT) + ); + } + + // Load root docs + const readmePath = path.join(dir, 'README.ai.md'); + if (fs.existsSync(readmePath)) { + result.rootReadme = truncateContent( + fs.readFileSync(readmePath, 'utf-8'), + ROOT_DOC_LIMIT + ); + } + + const pathFilePath = path.join(dir, 'PATH.ai.md'); + if (fs.existsSync(pathFilePath)) { + result.rootPath = truncateContent( + fs.readFileSync(pathFilePath, 'utf-8'), + ROOT_DOC_LIMIT + ); + } + + result.rootDir = dir; + return result; } -export function assembleWorkspacePrompt(files: AssistantWorkspaceFiles): string { - const order: Array = ['claude', 'soul', 'user', 'memory']; - const sections: string[] = []; +// ========================================== +// Budget-Aware Prompt Assembly (V2) +// ========================================== - for (const key of order) { - const content = files[key]; - if (content) { - sections.push(`<${key}>\n${content}\n`); +interface PromptSection { + tag: string; + content: string; + priority: number; // lower = higher priority + maxSize: number; +} + +export function assembleWorkspacePrompt(files: AssistantWorkspaceFilesV2, retrievalResults?: SearchResult[]): string { + const sections: PromptSection[] = []; + + // Priority 1: Identity (claude + soul + user) — never drop claude + if (files.claude) { + sections.push({ tag: 'claude', content: files.claude, priority: 1, maxSize: PER_FILE_LIMIT }); + } + if (files.soul) { + sections.push({ tag: 'soul', content: files.soul, priority: 1, maxSize: PER_FILE_LIMIT }); + } + if (files.user) { + sections.push({ tag: 'user', content: files.user, priority: 1, maxSize: PER_FILE_LIMIT }); + } + + // Priority 2: Long-term memory + if (files.memory) { + sections.push({ tag: 'memory', content: files.memory, priority: 2, maxSize: PER_FILE_LIMIT }); + } + + // Priority 3: Daily memories + if (files.dailyMemories && files.dailyMemories.length > 0) { + for (let i = 0; i < files.dailyMemories.length; i++) { + sections.push({ + tag: `daily-memory-${i}`, + content: files.dailyMemories[i], + priority: 3, + maxSize: DAILY_MEMORY_LIMIT, + }); + } + } + + // Priority 4: Root docs + if (files.rootReadme) { + sections.push({ tag: 'workspace-readme', content: files.rootReadme, priority: 4, maxSize: ROOT_DOC_LIMIT }); + } + if (files.rootPath) { + sections.push({ tag: 'workspace-path', content: files.rootPath, priority: 4, maxSize: ROOT_DOC_LIMIT }); + } + + // Priority 5: Retrieval results + if (retrievalResults && retrievalResults.length > 0) { + const results = retrievalResults.slice(0, MAX_RETRIEVAL_RESULTS); + for (const result of results) { + const content = `**${result.path}** (${result.source}, score: ${result.score.toFixed(1)})\n${result.heading ? `### ${result.heading}\n` : ''}${result.snippet}`; + sections.push({ + tag: 'retrieval-result', + content: truncateContent(content, RETRIEVAL_RESULT_LIMIT), + priority: 5, + maxSize: RETRIEVAL_RESULT_LIMIT, + }); } } if (sections.length === 0) return ''; - let prompt = `\n${sections.join('\n\n')}\n`; + // Sort by priority + sections.sort((a, b) => a.priority - b.priority); - if (prompt.length > TOTAL_PROMPT_LIMIT) { - prompt = prompt.slice(0, TOTAL_PROMPT_LIMIT); + // Budget-aware assembly + let totalSize = 0; + const included: string[] = []; + const wrapperOverhead = 50; // tags + + for (const section of sections) { + const sectionContent = truncateContent(section.content, section.maxSize); + const sectionSize = sectionContent.length + section.tag.length * 2 + 10; // tag overhead + + if (totalSize + sectionSize + wrapperOverhead > TOTAL_PROMPT_LIMIT) { + // Drop low-priority content on overflow, but never drop claude.md + if (section.tag === 'claude') { + // Force include claude.md even on overflow + included.push(`<${section.tag}>\n${sectionContent}\n`); + totalSize += sectionSize; + } + // Skip other sections that don't fit + continue; + } + + included.push(`<${section.tag}>\n${sectionContent}\n`); + totalSize += sectionSize; } - return prompt; + if (included.length === 0) return ''; + + return `\n${included.join('\n\n')}\n`; } +// ========================================== +// State Management +// ========================================== + export function loadState(dir: string): AssistantWorkspaceState { try { const statePath = path.join(dir, STATE_DIR, STATE_FILE); const raw = fs.readFileSync(statePath, 'utf-8'); - return JSON.parse(raw) as AssistantWorkspaceState; + const state = JSON.parse(raw) as AssistantWorkspaceState; + + // Auto-migrate on load + if (state.schemaVersion < 2) { + migrateStateV1ToV2(dir); + return loadState(dir); // Reload after migration + } + + return state; } catch { return { ...DEFAULT_STATE }; } @@ -160,6 +483,10 @@ export function needsDailyCheckIn(state: AssistantWorkspaceState): boolean { return state.lastCheckInDate !== new Date().toISOString().slice(0, 10); } +// ========================================== +// Directory Docs (legacy — kept for backward compatibility) +// ========================================== + export function generateDirectoryDocs(dir: string): string[] { const generated: string[] = []; diff --git a/src/lib/workspace-config.ts b/src/lib/workspace-config.ts new file mode 100644 index 0000000..8634d84 --- /dev/null +++ b/src/lib/workspace-config.ts @@ -0,0 +1,118 @@ +import fs from 'fs'; +import path from 'path'; +import { AssistantWorkspaceConfig } from '@/types'; + +const CONFIG_FILE = '.assistant/config.json'; + +export const DEFAULT_CONFIG: AssistantWorkspaceConfig = { + workspaceType: 'general', + organizationStyle: 'mixed', + captureDefault: 'Inbox', + archivePolicy: { + completedTaskArchiveAfterDays: 30, + closedProjectArchive: true, + dailyMemoryRetentionDays: 30, + }, + ignore: [ + '.obsidian/**', + '.trash/**', + '**/*.png', + '**/*.jpg', + '**/*.jpeg', + '**/*.gif', + '**/*.mp4', + 'node_modules/**', + '.git/**', + ], + index: { + maxFileSizeKB: 512, + chunkSize: 1200, + chunkOverlap: 150, + maxDepth: 8, + includeExtensions: ['.md', '.txt', '.markdown'], + }, +}; + +function deepMerge>(defaults: T, overrides: Partial): T { + const result = { ...defaults }; + for (const key of Object.keys(overrides) as (keyof T)[]) { + const val = overrides[key]; + if ( + val !== undefined && + val !== null && + typeof val === 'object' && + !Array.isArray(val) && + typeof defaults[key] === 'object' && + !Array.isArray(defaults[key]) + ) { + result[key] = deepMerge( + defaults[key] as Record, + val as Record, + ) as T[keyof T]; + } else if (val !== undefined) { + result[key] = val as T[keyof T]; + } + } + return result; +} + +export function loadConfig(dir: string): AssistantWorkspaceConfig { + const configPath = path.join(dir, CONFIG_FILE); + try { + const raw = fs.readFileSync(configPath, 'utf-8'); + const parsed = JSON.parse(raw) as Partial; + return deepMerge(DEFAULT_CONFIG as unknown as Record, parsed as unknown as Record) as unknown as AssistantWorkspaceConfig; + } catch { + return { ...DEFAULT_CONFIG }; + } +} + +export function saveConfig(dir: string, config: AssistantWorkspaceConfig): void { + const configPath = path.join(dir, CONFIG_FILE); + const configDir = path.dirname(configPath); + if (!fs.existsSync(configDir)) { + fs.mkdirSync(configDir, { recursive: true }); + } + fs.writeFileSync(configPath, JSON.stringify(config, null, 2) + '\n', 'utf-8'); +} + +function globToRegex(pattern: string): RegExp { + let regex = ''; + let i = 0; + while (i < pattern.length) { + const ch = pattern[i]; + if (ch === '*' && pattern[i + 1] === '*') { + // ** matches any path segment(s) + if (pattern[i + 2] === '/') { + regex += '(?:.+/)?'; + i += 3; + } else { + regex += '.*'; + i += 2; + } + } else if (ch === '*') { + // * matches anything except / + regex += '[^/]*'; + i++; + } else if (ch === '?') { + regex += '[^/]'; + i++; + } else if (ch === '.') { + regex += '\\.'; + i++; + } else { + regex += ch; + i++; + } + } + return new RegExp(`^${regex}$`); +} + +export function shouldIgnore(filePath: string, config: AssistantWorkspaceConfig): boolean { + // Normalize to forward slashes for consistent matching + const normalized = filePath.replace(/\\/g, '/'); + return config.ignore.some((pattern) => { + const re = globToRegex(pattern); + return re.test(normalized); + }); +} diff --git a/src/lib/workspace-indexer.ts b/src/lib/workspace-indexer.ts new file mode 100644 index 0000000..0153f9b --- /dev/null +++ b/src/lib/workspace-indexer.ts @@ -0,0 +1,427 @@ +import fs from 'fs'; +import path from 'path'; +import crypto from 'crypto'; +import type { ManifestEntry, ChunkEntry, HotsetFile, AssistantWorkspaceConfig } from '@/types'; +import { loadConfig, shouldIgnore } from '@/lib/workspace-config'; +import { loadTaxonomy, classifyPath } from '@/lib/workspace-taxonomy'; + +export const INDEX_DIR = '.assistant/index'; + +export function computeFileHash(content: string): string { + return crypto.createHash('md5').update(content).digest('hex'); +} + +export function chunkMarkdown( + content: string, + chunkSize = 1200, + overlap = 150, +): Array<{ heading: string; text: string; startLine: number; endLine: number }> { + const lines = content.split('\n'); + const sections: Array<{ heading: string; lines: string[]; startLine: number }> = []; + + let currentHeading = ''; + let currentLines: string[] = []; + let sectionStart = 0; + + for (let i = 0; i < lines.length; i++) { + const line = lines[i]; + if (line.match(/^#{1,6}\s/)) { + if (currentLines.length > 0) { + sections.push({ heading: currentHeading, lines: currentLines, startLine: sectionStart }); + } + currentHeading = line.replace(/^#{1,6}\s+/, '').trim(); + currentLines = [line]; + sectionStart = i; + } else { + currentLines.push(line); + } + } + if (currentLines.length > 0) { + sections.push({ heading: currentHeading, lines: currentLines, startLine: sectionStart }); + } + + const chunks: Array<{ heading: string; text: string; startLine: number; endLine: number }> = []; + + for (const section of sections) { + const sectionText = section.lines.join('\n'); + if (sectionText.length <= chunkSize) { + chunks.push({ + heading: section.heading, + text: sectionText, + startLine: section.startLine, + endLine: section.startLine + section.lines.length - 1, + }); + } else { + // Split large sections with overlap + let charPos = 0; + let lineIdx = 0; + + while (charPos < sectionText.length) { + const end = Math.min(charPos + chunkSize, sectionText.length); + const chunkText = sectionText.slice(charPos, end); + + // Determine line range for this chunk + const chunkLines = chunkText.split('\n'); + const startLine = section.startLine + lineIdx; + const endLine = startLine + chunkLines.length - 1; + + chunks.push({ + heading: section.heading, + text: chunkText, + startLine, + endLine, + }); + + if (end >= sectionText.length) break; + + // Advance by chunkSize - overlap + const advance = chunkSize - overlap; + const advancedText = sectionText.slice(charPos, charPos + advance); + const advancedLines = advancedText.split('\n'); + lineIdx += advancedLines.length - 1; + charPos += advance; + } + } + } + + return chunks; +} + +export function extractMarkdownMeta(content: string): { + title: string; + tags: string[]; + headings: string[]; + aliases: string[]; +} { + const lines = content.split('\n'); + let title = ''; + const tags: string[] = []; + const headings: string[] = []; + const aliases: string[] = []; + + // Parse YAML frontmatter + if (lines[0]?.trim() === '---') { + let fmEnd = -1; + for (let i = 1; i < lines.length; i++) { + if (lines[i].trim() === '---') { + fmEnd = i; + break; + } + } + if (fmEnd > 0) { + const fmLines = lines.slice(1, fmEnd); + let currentKey = ''; + for (const fmLine of fmLines) { + const kvMatch = fmLine.match(/^(\w+)\s*:\s*(.*)$/); + if (kvMatch) { + currentKey = kvMatch[1]; + const value = kvMatch[2].trim(); + if (currentKey === 'tags') { + // Inline array: tags: [a, b, c] + const inlineMatch = value.match(/^\[(.+)\]$/); + if (inlineMatch) { + tags.push(...inlineMatch[1].split(',').map((t) => t.trim().replace(/^["']|["']$/g, ''))); + currentKey = ''; + } + // If value is non-empty but not array syntax, treat as single tag + else if (value) { + tags.push(value.replace(/^["']|["']$/g, '')); + currentKey = ''; + } + } else if (currentKey === 'aliases') { + const inlineMatch = value.match(/^\[(.+)\]$/); + if (inlineMatch) { + aliases.push(...inlineMatch[1].split(',').map((a) => a.trim().replace(/^["']|["']$/g, ''))); + currentKey = ''; + } else if (value) { + aliases.push(value.replace(/^["']|["']$/g, '')); + currentKey = ''; + } + } else { + currentKey = ''; + } + } else if (currentKey) { + // List item under current key + const listMatch = fmLine.match(/^\s*-\s+(.+)$/); + if (listMatch) { + const item = listMatch[1].trim().replace(/^["']|["']$/g, ''); + if (currentKey === 'tags') tags.push(item); + else if (currentKey === 'aliases') aliases.push(item); + } else { + currentKey = ''; + } + } + } + } + } + + // Extract headings and title + for (const line of lines) { + const headingMatch = line.match(/^(#{1,6})\s+(.+)$/); + if (headingMatch) { + headings.push(headingMatch[2].trim()); + if (!title && headingMatch[1] === '#') { + title = headingMatch[2].trim(); + } + } + } + + // Extract inline #tags from content (not inside code blocks) + let inCodeBlock = false; + for (const line of lines) { + if (line.startsWith('```')) { + inCodeBlock = !inCodeBlock; + continue; + } + if (inCodeBlock) continue; + const tagMatches = line.match(/(?:^|\s)#([a-zA-Z][a-zA-Z0-9_-]*)/g); + if (tagMatches) { + for (const match of tagMatches) { + const tag = match.trim().slice(1); // remove # + // Skip if it looks like a heading (line starts with #) + if (!line.match(/^#{1,6}\s/) && !tags.includes(tag)) { + tags.push(tag); + } + } + } + } + + return { title, tags, headings, aliases }; +} + +export function needsReindex(dir: string, filePath: string): boolean { + const manifestPath = path.join(dir, INDEX_DIR, 'manifest.jsonl'); + if (!fs.existsSync(manifestPath)) return true; + + let fileMtime: number; + try { + fileMtime = fs.statSync(path.join(dir, filePath)).mtimeMs; + } catch { + return true; + } + + const manifest = loadManifest(dir); + const entry = manifest.find((e) => e.path === filePath); + if (!entry) return true; + + return fileMtime > entry.mtime; +} + +export function indexFile( + dir: string, + filePath: string, +): { manifest: ManifestEntry; chunks: ChunkEntry[] } { + const fullPath = path.join(dir, filePath); + const content = fs.readFileSync(fullPath, 'utf-8'); + const stat = fs.statSync(fullPath); + const hash = computeFileHash(content); + const meta = extractMarkdownMeta(content); + const noteId = computeFileHash(filePath); + + const taxonomy = loadTaxonomy(dir); + const category = classifyPath(filePath, taxonomy); + const categoryIds = category ? [category.id] : []; + + const config = loadConfig(dir); + const rawChunks = chunkMarkdown(content, config.index.chunkSize, config.index.chunkOverlap); + + const chunks: ChunkEntry[] = rawChunks.map((c, i) => ({ + chunkId: `${noteId}-${i}`, + noteId, + path: filePath, + heading: c.heading, + text: c.text, + startLine: c.startLine, + endLine: c.endLine, + })); + + const manifest: ManifestEntry = { + noteId, + path: filePath, + title: meta.title || path.basename(filePath, path.extname(filePath)), + aliases: meta.aliases, + tags: meta.tags, + headings: meta.headings, + mtime: stat.mtimeMs, + size: stat.size, + hash, + summary: content.slice(0, 200).replace(/\n/g, ' ').trim(), + categoryIds, + }; + + return { manifest, chunks }; +} + +function walkDir( + dir: string, + baseDir: string, + config: AssistantWorkspaceConfig, + depth: number, +): string[] { + if (depth > config.index.maxDepth) return []; + + let entries: fs.Dirent[]; + try { + entries = fs.readdirSync(dir, { withFileTypes: true }); + } catch { + return []; + } + + const files: string[] = []; + const maxBytes = config.index.maxFileSizeKB * 1024; + const extensions = new Set(config.index.includeExtensions); + + for (const entry of entries) { + const fullPath = path.join(dir, entry.name); + const relPath = path.relative(baseDir, fullPath).replace(/\\/g, '/'); + + if (shouldIgnore(relPath, config)) continue; + + if (entry.isDirectory()) { + files.push(...walkDir(fullPath, baseDir, config, depth + 1)); + } else if (entry.isFile()) { + const ext = path.extname(entry.name).toLowerCase(); + if (!extensions.has(ext)) continue; + + try { + const stat = fs.statSync(fullPath); + if (stat.size > maxBytes) continue; + } catch { + continue; + } + + files.push(relPath); + } + } + + return files; +} + +export function indexWorkspace(dir: string, options?: { force?: boolean }): { fileCount: number; chunkCount: number } { + const config = loadConfig(dir); + const force = options?.force ?? false; + + const indexDir = path.join(dir, INDEX_DIR); + if (!fs.existsSync(indexDir)) { + fs.mkdirSync(indexDir, { recursive: true }); + } + + const files = walkDir(dir, dir, config, 0); + const fileSet = new Set(files); + + // Load existing index for incremental update + const existingManifest = force ? [] : loadManifest(dir); + const existingChunks = force ? [] : loadChunks(dir); + + // Build lookup maps from existing index + const existingByPath = new Map(); + for (const entry of existingManifest) { + existingByPath.set(entry.path, entry); + } + const existingChunksByNote = new Map(); + for (const chunk of existingChunks) { + const list = existingChunksByNote.get(chunk.noteId) || []; + list.push(chunk); + existingChunksByNote.set(chunk.noteId, list); + } + + const manifests: ManifestEntry[] = []; + const allChunks: ChunkEntry[] = []; + let reindexed = 0; + + for (const filePath of files) { + const existing = existingByPath.get(filePath); + + // Skip files that haven't changed (incremental) + if (!force && existing) { + try { + const stat = fs.statSync(path.join(dir, filePath)); + if (stat.mtimeMs <= existing.mtime) { + // Reuse existing manifest and chunks + manifests.push(existing); + const existingFileChunks = existingChunksByNote.get(existing.noteId) || []; + allChunks.push(...existingFileChunks); + continue; + } + } catch { + // File stat failed, re-index it + } + } + + try { + const { manifest, chunks } = indexFile(dir, filePath); + manifests.push(manifest); + allChunks.push(...chunks); + reindexed++; + } catch { + // Skip files that fail to index + } + } + + // Only write if something changed (or force) + if (force || reindexed > 0 || manifests.length !== existingManifest.length) { + const manifestLines = manifests.map((m) => JSON.stringify(m)).join('\n'); + fs.writeFileSync(path.join(indexDir, 'manifest.jsonl'), manifestLines + '\n', 'utf-8'); + + const chunkLines = allChunks.map((c) => JSON.stringify(c)).join('\n'); + fs.writeFileSync(path.join(indexDir, 'chunks.jsonl'), chunkLines + '\n', 'utf-8'); + } + + return { fileCount: manifests.length, chunkCount: allChunks.length }; +} + +export function loadManifest(dir: string): ManifestEntry[] { + const manifestPath = path.join(dir, INDEX_DIR, 'manifest.jsonl'); + try { + const content = fs.readFileSync(manifestPath, 'utf-8').trim(); + if (!content) return []; + return content.split('\n').map((line) => JSON.parse(line) as ManifestEntry); + } catch { + return []; + } +} + +export function loadChunks(dir: string): ChunkEntry[] { + const chunksPath = path.join(dir, INDEX_DIR, 'chunks.jsonl'); + try { + const content = fs.readFileSync(chunksPath, 'utf-8').trim(); + if (!content) return []; + return content.split('\n').map((line) => JSON.parse(line) as ChunkEntry); + } catch { + return []; + } +} + +export function getIndexStats(dir: string): { + fileCount: number; + chunkCount: number; + lastIndexed: number; + staleCount: number; +} { + const manifest = loadManifest(dir); + const chunks = loadChunks(dir); + + let lastIndexed = 0; + let staleCount = 0; + + for (const entry of manifest) { + if (entry.mtime > lastIndexed) { + lastIndexed = entry.mtime; + } + try { + const stat = fs.statSync(path.join(dir, entry.path)); + if (stat.mtimeMs > entry.mtime) { + staleCount++; + } + } catch { + staleCount++; // File deleted + } + } + + return { + fileCount: manifest.length, + chunkCount: chunks.length, + lastIndexed, + staleCount, + }; +} diff --git a/src/lib/workspace-organizer.ts b/src/lib/workspace-organizer.ts new file mode 100644 index 0000000..f930e8d --- /dev/null +++ b/src/lib/workspace-organizer.ts @@ -0,0 +1,294 @@ +import fs from 'fs'; +import path from 'path'; +import type { TaxonomyCategory, TaxonomyFile } from '@/types'; +import { loadConfig } from '@/lib/workspace-config'; +import { loadTaxonomy, saveTaxonomy, classifyPath } from '@/lib/workspace-taxonomy'; +import { loadManifest } from '@/lib/workspace-indexer'; + +// --------------------------------------------------------------------------- +// Helpers +// --------------------------------------------------------------------------- + +/** + * Assert that `target` resolves to a path inside `workspaceDir`. + * Rejects absolute paths, `../` traversals, and symlink escapes. + */ +export function assertContained(workspaceDir: string, relativePath: string): string { + // Reject absolute paths and obvious traversal attempts early + if (path.isAbsolute(relativePath)) { + throw new Error(`Absolute paths are not allowed: ${relativePath}`); + } + if (relativePath.startsWith('~')) { + throw new Error(`Home-relative paths are not allowed: ${relativePath}`); + } + + const resolved = path.resolve(workspaceDir, relativePath); + const normalizedWorkspace = path.resolve(workspaceDir); + + // Must start with workspace dir + separator (or equal it) + if (resolved !== normalizedWorkspace && !resolved.startsWith(normalizedWorkspace + path.sep)) { + throw new Error(`Path escapes workspace boundary: ${relativePath}`); + } + + // If the resolved path exists, check it's not a symlink pointing outside + try { + const real = fs.realpathSync(resolved); + if (real !== normalizedWorkspace && !real.startsWith(normalizedWorkspace + path.sep)) { + throw new Error(`Symlink target escapes workspace boundary: ${relativePath}`); + } + } catch (e) { + // File doesn't exist yet — that's fine for targets being created + if ((e as NodeJS.ErrnoException).code !== 'ENOENT') throw e; + } + + return resolved; +} + +function sanitizeFilename(title: string): string { + return title + .replace(/[<>:"/\\|?*\x00-\x1f]/g, '') + .replace(/\s+/g, '-') + .replace(/-+/g, '-') + .replace(/^-|-$/g, '') + .slice(0, 200); +} + +function ensureDir(dirPath: string): void { + if (!fs.existsSync(dirPath)) { + fs.mkdirSync(dirPath, { recursive: true }); + } +} + +function parseDateFromFilename(filename: string): Date | null { + const match = filename.match(/^(\d{4}-\d{2}-\d{2})\.md$/); + if (!match) return null; + const d = new Date(match[1] + 'T00:00:00'); + return isNaN(d.getTime()) ? null : d; +} + +function daysBetween(a: Date, b: Date): number { + const msPerDay = 86_400_000; + return Math.floor(Math.abs(a.getTime() - b.getTime()) / msPerDay); +} + +// --------------------------------------------------------------------------- +// 1. Capture +// --------------------------------------------------------------------------- + +export function captureNote(dir: string, title: string, content: string): string { + const config = loadConfig(dir); + const captureDir = config.captureDefault || 'Inbox'; + + // Validate captureDefault is a safe relative path within workspace + const filename = sanitizeFilename(title) + '.md'; + const relativePath = path.join(captureDir, filename); + const absTarget = assertContained(dir, relativePath); + + ensureDir(path.dirname(absTarget)); + fs.writeFileSync(absTarget, content, 'utf-8'); + return absTarget; +} + +// --------------------------------------------------------------------------- +// 2. Classify & Suggest +// --------------------------------------------------------------------------- + +export function classifyAndSuggest( + dir: string, + filePath: string, +): { suggestedCategory: TaxonomyCategory | null; confidence: number; suggestedPath: string } { + const taxonomy = loadTaxonomy(dir); + const category = classifyPath(filePath, taxonomy); + + if (category) { + const primaryPath = category.paths[0] || 'Inbox/'; + const basename = path.basename(filePath); + return { + suggestedCategory: category, + confidence: category.confidence, + suggestedPath: path.join(primaryPath, basename), + }; + } + + // No good match — suggest Inbox + const basename = path.basename(filePath); + return { + suggestedCategory: null, + confidence: 0, + suggestedPath: path.join('Inbox', basename), + }; +} + +// --------------------------------------------------------------------------- +// 3. Move File +// --------------------------------------------------------------------------- + +export function moveFile(dir: string, fromPath: string, toPath: string): void { + const absFrom = assertContained(dir, fromPath); + const absTo = assertContained(dir, toPath); + ensureDir(path.dirname(absTo)); + fs.renameSync(absFrom, absTo); +} + +// --------------------------------------------------------------------------- +// 4. Archive Daily Memories +// --------------------------------------------------------------------------- + +export function archiveDailyMemories( + dir: string, +): { archived: number; retained: number } { + const config = loadConfig(dir); + const retentionDays = config.archivePolicy.dailyMemoryRetentionDays; + const dailyDir = path.join(dir, 'memory', 'daily'); + + if (!fs.existsSync(dailyDir)) { + return { archived: 0, retained: 0 }; + } + + const now = new Date(); + let archived = 0; + let retained = 0; + + const entries = fs.readdirSync(dailyDir); + const archivePath = path.join(dailyDir, 'archive.md'); + + for (const entry of entries) { + if (entry === 'archive.md') continue; + const fileDate = parseDateFromFilename(entry); + if (!fileDate) { + retained++; + continue; + } + + if (daysBetween(now, fileDate) > retentionDays) { + // Read content and build summary line + const fullPath = path.join(dailyDir, entry); + const content = fs.readFileSync(fullPath, 'utf-8'); + const firstLine = content.split('\n').find((l) => l.trim().length > 0) || entry; + const summaryLine = `- **${entry}**: ${firstLine.replace(/^#+\s*/, '').trim()}\n`; + + // Append to archive + fs.appendFileSync(archivePath, summaryLine, 'utf-8'); + fs.unlinkSync(fullPath); + archived++; + } else { + retained++; + } + } + + return { archived, retained }; +} + +// --------------------------------------------------------------------------- +// 5. Promote Daily to Long-Term +// --------------------------------------------------------------------------- + +export function promoteDailyToLongTerm(dir: string, date: string): boolean { + const dailyFile = path.join(dir, 'memory', 'daily', `${date}.md`); + if (!fs.existsSync(dailyFile)) return false; + + const content = fs.readFileSync(dailyFile, 'utf-8'); + + // Skip if already promoted (idempotency marker) + if (content.includes('')) return false; + + // Look for promotion-worthy sections + const sectionPattern = /^##\s+(Candidate Long-Term Memory|To Remember)\s*$/im; + const match = content.match(sectionPattern); + if (!match) return false; + + // Extract content from the matched section until the next ## heading or EOF + const sectionStart = match.index! + match[0].length; + const rest = content.slice(sectionStart); + const nextHeading = rest.search(/^##\s+/m); + const sectionContent = + nextHeading >= 0 ? rest.slice(0, nextHeading).trim() : rest.trim(); + + // Only promote if non-trivial + if (sectionContent.length <= 50) return false; + + // Check for duplicates in memory.md + const memoryFile = path.join(dir, 'memory.md'); + let existingMemory = ''; + try { existingMemory = fs.readFileSync(memoryFile, 'utf-8'); } catch { /* new file */ } + + if (existingMemory.includes(`## Promoted from ${date}`)) return false; + + const promotionBlock = `\n\n## Promoted from ${date}\n\n${sectionContent}\n`; + fs.appendFileSync(memoryFile, promotionBlock, 'utf-8'); + + // Mark daily file as promoted so it won't be promoted again + fs.appendFileSync(dailyFile, '\n\n', 'utf-8'); + return true; +} + +// --------------------------------------------------------------------------- +// 6. Suggest Taxonomy Evolution +// --------------------------------------------------------------------------- + +export function suggestTaxonomyEvolution( + dir: string, +): Array<{ type: 'new_category' | 'merge' | 'archive_category'; description: string; confidence: number }> { + const taxonomy = loadTaxonomy(dir); + const manifest = loadManifest(dir); + const suggestions: Array<{ + type: 'new_category' | 'merge' | 'archive_category'; + description: string; + confidence: number; + }> = []; + + // Count files per category + const categoryFileCounts = new Map(); + for (const cat of taxonomy.categories) { + categoryFileCounts.set(cat.id, 0); + } + + const uncategorized: string[] = []; + + for (const entry of manifest) { + const category = classifyPath(entry.path, taxonomy); + if (category) { + categoryFileCounts.set( + category.id, + (categoryFileCounts.get(category.id) || 0) + 1, + ); + } else { + uncategorized.push(entry.path); + } + } + + // Files without a good category → suggest new category + if (uncategorized.length > 0) { + // Group by top-level directory + const dirGroups = new Map(); + for (const p of uncategorized) { + const topDir = p.split('/')[0] || p; + dirGroups.set(topDir, (dirGroups.get(topDir) || 0) + 1); + } + for (const [dirName, count] of dirGroups) { + if (count >= 2) { + suggestions.push({ + type: 'new_category', + description: `Create category for "${dirName}" (${count} uncategorized files)`, + confidence: Math.min(0.9, 0.4 + count * 0.1), + }); + } + } + } + + // Categories with 0 files → suggest archive + for (const [catId, count] of categoryFileCounts) { + if (count === 0) { + const cat = taxonomy.categories.find((c) => c.id === catId); + suggestions.push({ + type: 'archive_category', + description: `Archive empty category "${cat?.label || catId}" (0 files)`, + confidence: 0.7, + }); + } + } + + // Sort by confidence descending + suggestions.sort((a, b) => b.confidence - a.confidence); + return suggestions; +} diff --git a/src/lib/workspace-retrieval.ts b/src/lib/workspace-retrieval.ts new file mode 100644 index 0000000..5c9c801 --- /dev/null +++ b/src/lib/workspace-retrieval.ts @@ -0,0 +1,292 @@ +/** + * Keyword-based search over the workspace index. + * Pure text matching — no vector DB required. + */ + +import fs from 'fs'; +import path from 'path'; +import type { ManifestEntry, ChunkEntry, HotsetFile, SearchResult } from '@/types'; +import { loadManifest, loadChunks, INDEX_DIR } from '@/lib/workspace-indexer'; + +// --------------------------------------------------------------------------- +// Stop words +// --------------------------------------------------------------------------- + +const STOP_WORDS = new Set([ + 'the', 'a', 'an', 'is', 'are', 'was', 'were', 'be', 'been', 'being', + 'have', 'has', 'had', 'do', 'does', 'did', 'will', 'would', 'shall', + 'should', 'may', 'might', 'can', 'could', 'of', 'at', 'by', 'for', + 'with', 'about', 'against', 'between', 'through', 'during', 'before', + 'after', 'above', 'below', 'to', 'from', 'up', 'down', 'in', 'out', + 'on', 'off', 'over', 'under', 'again', 'further', 'then', 'once', + 'and', 'but', 'or', 'nor', 'not', 'so', 'if', 'this', 'that', 'these', + 'those', 'it', 'its', 'i', 'me', 'my', 'we', 'our', 'you', 'your', + 'he', 'his', 'she', 'her', 'they', 'their', 'what', 'which', 'who', 'whom', +]); + +// --------------------------------------------------------------------------- +// CJK detection & bigram tokenization +// --------------------------------------------------------------------------- + +const CJK_RANGE = /[\u4e00-\u9fff\u3400-\u4dbf\u3000-\u303f\uf900-\ufaff]/; + +function isCJK(ch: string): boolean { + return CJK_RANGE.test(ch); +} + +/** Split a CJK run into overlapping bigrams (+ individual chars as fallback). */ +function cjkBigrams(text: string): string[] { + const chars = [...text]; + const tokens: string[] = []; + for (let i = 0; i < chars.length; i++) { + tokens.push(chars[i]); // unigram + if (i + 1 < chars.length) { + tokens.push(chars[i] + chars[i + 1]); // bigram + } + } + return tokens; +} + +// --------------------------------------------------------------------------- +// Query parsing +// --------------------------------------------------------------------------- + +export function parseQuery(query: string): string[] { + const lower = query.toLowerCase(); + const tokens: string[] = []; + + // Split into runs of CJK vs non-CJK + let buf = ''; + let wasCJK = false; + + for (const ch of lower) { + const nowCJK = isCJK(ch); + if (buf.length > 0 && nowCJK !== wasCJK) { + // flush previous run + if (wasCJK) { + tokens.push(...cjkBigrams(buf)); + } else { + const words = buf.split(/[\s\p{P}]+/u).filter(t => t.length > 0 && !STOP_WORDS.has(t)); + tokens.push(...words); + } + buf = ''; + } + // Skip whitespace/punctuation between runs but keep CJK chars + if (nowCJK || !/[\s\p{P}]/u.test(ch)) { + buf += ch; + } else if (buf.length > 0 && !wasCJK) { + buf += ch; // let split handle it + } + wasCJK = nowCJK; + } + + // flush remainder + if (buf.length > 0) { + if (wasCJK) { + tokens.push(...cjkBigrams(buf)); + } else { + const words = buf.split(/[\s\p{P}]+/u).filter(t => t.length > 0 && !STOP_WORDS.has(t)); + tokens.push(...words); + } + } + + return [...new Set(tokens)]; +} + +// --------------------------------------------------------------------------- +// Manifest scoring +// --------------------------------------------------------------------------- + +export function scoreManifest( + entry: ManifestEntry, + keywords: string[], +): { score: number; source: SearchResult['source'] } { + let score = 0; + let bestWeight = 0; + let source: SearchResult['source'] = 'content'; + + const titleLower = entry.title.toLowerCase(); + const pathLower = entry.path.toLowerCase(); + const headingsLower = entry.headings.map(h => h.toLowerCase()); + const tagsLower = entry.tags.map(t => t.toLowerCase()); + const aliasesLower = entry.aliases.map(a => a.toLowerCase()); + + for (const kw of keywords) { + if (titleLower.includes(kw)) { + score += 10; + if (10 > bestWeight) { bestWeight = 10; source = 'title'; } + } + if (tagsLower.some(t => t.includes(kw))) { + score += 8; + if (8 > bestWeight) { bestWeight = 8; source = 'tag'; } + } + if (aliasesLower.some(a => a.includes(kw))) { + score += 7; + } + if (headingsLower.some(h => h.includes(kw))) { + score += 5; + if (5 > bestWeight && bestWeight < 8) { bestWeight = 5; source = 'heading'; } + } + if (pathLower.includes(kw)) { + score += 3; + } + } + + return { score, source }; +} + +// --------------------------------------------------------------------------- +// Chunk scoring +// --------------------------------------------------------------------------- + +export function scoreChunk(chunk: ChunkEntry, keywords: string[]): number { + let score = 0; + const textLower = chunk.text.toLowerCase(); + const headingLower = chunk.heading.toLowerCase(); + + for (const kw of keywords) { + // Count occurrences in text + let idx = 0; + while ((idx = textLower.indexOf(kw, idx)) !== -1) { + score += 1; + idx += kw.length; + } + // Heading matches get 2x weight + idx = 0; + while ((idx = headingLower.indexOf(kw, idx)) !== -1) { + score += 2; + idx += kw.length; + } + } + + return score; +} + +// --------------------------------------------------------------------------- +// Main search +// --------------------------------------------------------------------------- + +export function searchWorkspace( + dir: string, + query: string, + options?: { limit?: number }, +): SearchResult[] { + const limit = options?.limit ?? 5; + const keywords = parseQuery(query); + if (keywords.length === 0) return []; + + const manifest = loadManifest(dir); + + // Load hotset for score boosting + const hotset = loadHotset(dir); + const pinnedSet = new Set(hotset.pinned); + const frequentMap = new Map(); + for (const f of hotset.frequent) { + frequentMap.set(f.path, f.count); + } + + // Score every manifest entry + const scored = manifest.map(entry => { + const { score: baseScore, source } = scoreManifest(entry, keywords); + let score = baseScore; + + // Boost pinned files + if (pinnedSet.has(entry.path)) { + score += 5; + } + // Boost frequently accessed files (diminishing returns, cap at +4) + const freq = frequentMap.get(entry.path); + if (freq) { + score += Math.min(4, Math.log2(freq + 1)); + } + + return { entry, score, source }; + }); + + // Take top 2*limit candidates by manifest score + scored.sort((a, b) => b.score - a.score); + const candidates = scored.filter(s => s.score > 0).slice(0, limit * 2); + + if (candidates.length === 0) return []; + + // Load all chunks once, group by noteId + const allChunks = loadChunks(dir); + const chunksByNote = new Map(); + for (const chunk of allChunks) { + const list = chunksByNote.get(chunk.noteId) || []; + list.push(chunk); + chunksByNote.set(chunk.noteId, list); + } + + // For each candidate, find the best chunk + const results: SearchResult[] = []; + + for (const candidate of candidates) { + const chunks = chunksByNote.get(candidate.entry.noteId) || []; + let bestChunkScore = 0; + let bestChunk: ChunkEntry | null = null; + + for (const chunk of chunks) { + const cs = scoreChunk(chunk, keywords); + if (cs > bestChunkScore) { + bestChunkScore = cs; + bestChunk = chunk; + } + } + + const totalScore = candidate.score + bestChunkScore; + const snippet = bestChunk + ? bestChunk.text.slice(0, 300) + : candidate.entry.summary.slice(0, 300); + const heading = bestChunk?.heading ?? candidate.entry.headings[0] ?? ''; + + results.push({ + path: candidate.entry.path, + heading, + snippet, + score: totalScore, + source: bestChunkScore > candidate.score ? 'content' : candidate.source, + }); + } + + results.sort((a, b) => b.score - a.score); + return results.slice(0, limit); +} + +// --------------------------------------------------------------------------- +// Hotset management +// --------------------------------------------------------------------------- + +export function loadHotset(dir: string): HotsetFile { + const hotsetPath = path.join(dir, INDEX_DIR, 'hotset.json'); + try { + const raw = fs.readFileSync(hotsetPath, 'utf-8'); + return JSON.parse(raw) as HotsetFile; + } catch { + return { pinned: [], frequent: [], lastUpdated: Date.now() }; + } +} + +export function updateHotset(dir: string, accessedPaths: string[]): void { + const hotset = loadHotset(dir); + const now = Date.now(); + + for (const p of accessedPaths) { + const existing = hotset.frequent.find(f => f.path === p); + if (existing) { + existing.count += 1; + existing.lastAccessed = now; + } else { + hotset.frequent.push({ path: p, count: 1, lastAccessed: now }); + } + } + + // Sort by count descending, cap at 20 + hotset.frequent.sort((a, b) => b.count - a.count); + hotset.frequent = hotset.frequent.slice(0, 20); + hotset.lastUpdated = now; + + const hotsetPath = path.join(dir, INDEX_DIR, 'hotset.json'); + fs.mkdirSync(path.dirname(hotsetPath), { recursive: true }); + fs.writeFileSync(hotsetPath, JSON.stringify(hotset, null, 2), 'utf-8'); +} diff --git a/src/lib/workspace-taxonomy.ts b/src/lib/workspace-taxonomy.ts new file mode 100644 index 0000000..a52b9f9 --- /dev/null +++ b/src/lib/workspace-taxonomy.ts @@ -0,0 +1,159 @@ +import fs from 'fs'; +import path from 'path'; +import type { TaxonomyCategory, TaxonomyFile } from '@/types'; + +const TAXONOMY_FILENAME = 'taxonomy.json'; +const ASSISTANT_DIR = '.assistant'; + +export const DEFAULT_TAXONOMY: TaxonomyFile = { + version: 1, + categories: [], + evolutionRules: { + allowAutoCreateCategory: false, + allowAutoArchive: false, + requireConfirmationForMoves: true, + }, +}; + +function getTaxonomyPath(dir: string): string { + return path.join(dir, ASSISTANT_DIR, TAXONOMY_FILENAME); +} + +export function loadTaxonomy(dir: string): TaxonomyFile { + const filePath = getTaxonomyPath(dir); + try { + const content = fs.readFileSync(filePath, 'utf-8'); + return JSON.parse(content) as TaxonomyFile; + } catch { + return { ...DEFAULT_TAXONOMY, categories: [] }; + } +} + +export function saveTaxonomy(dir: string, taxonomy: TaxonomyFile): void { + const filePath = getTaxonomyPath(dir); + const assistantDir = path.join(dir, ASSISTANT_DIR); + if (!fs.existsSync(assistantDir)) { + fs.mkdirSync(assistantDir, { recursive: true }); + } + fs.writeFileSync(filePath, JSON.stringify(taxonomy, null, 2), 'utf-8'); +} + +const ROLE_MAP: Record = { + notes: 'notes', + note: 'notes', + projects: 'project', + project: 'project', + journal: 'journal', + diary: 'journal', + daily: 'journal', + archive: 'archive', + archives: 'archive', + inbox: 'inbox', + templates: 'template', + template: 'template', + resources: 'resource', + assets: 'resource', + attachments: 'resource', + memory: 'memory', +}; + +// Names that are an exact canonical form get higher confidence +const EXACT_NAMES = new Set([ + 'notes', + 'projects', + 'journal', + 'archive', + 'archives', + 'inbox', + 'templates', + 'resources', + 'assets', + 'attachments', + 'memory', + 'daily', + 'diary', +]); + +function inferRole(dirName: string): { role: string; confidence: number } { + const lower = dirName.toLowerCase(); + const role = ROLE_MAP[lower]; + if (role) { + const confidence = EXACT_NAMES.has(lower) ? 0.9 : 0.6; + return { role, confidence }; + } + return { role: 'unknown', confidence: 0.3 }; +} + +export function inferTaxonomyFromDirs(dir: string): TaxonomyCategory[] { + let entries: fs.Dirent[]; + try { + entries = fs.readdirSync(dir, { withFileTypes: true }); + } catch { + return []; + } + + const categories: TaxonomyCategory[] = []; + + for (const entry of entries) { + if (!entry.isDirectory()) continue; + if (entry.name.startsWith('.')) continue; + + const { role, confidence } = inferRole(entry.name); + categories.push({ + id: entry.name.toLowerCase().replace(/[^a-z0-9-]/g, '-'), + label: entry.name, + paths: [entry.name + '/'], + role, + confidence, + source: 'learned', + description: role !== 'unknown' + ? `Auto-detected ${role} directory` + : `Unknown directory: ${entry.name}`, + createdBy: 'system', + }); + } + + return categories; +} + +export function classifyPath( + filePath: string, + taxonomy: TaxonomyFile, +): TaxonomyCategory | null { + const normalized = filePath.replace(/\\/g, '/'); + let bestMatch: TaxonomyCategory | null = null; + let bestLength = 0; + + for (const category of taxonomy.categories) { + for (const catPath of category.paths) { + const normalizedCatPath = catPath.replace(/\\/g, '/'); + if ( + normalized.startsWith(normalizedCatPath) || + normalized.startsWith('/' + normalizedCatPath) + ) { + if (normalizedCatPath.length > bestLength) { + bestLength = normalizedCatPath.length; + bestMatch = category; + } + } + } + } + + return bestMatch; +} + +export function suggestNewCategory(dirName: string): TaxonomyCategory { + const { role, confidence } = inferRole(dirName); + return { + id: dirName.toLowerCase().replace(/[^a-z0-9-]/g, '-'), + label: dirName, + paths: [dirName + '/'], + role, + confidence, + source: 'learned', + description: role !== 'unknown' + ? `Suggested ${role} category for "${dirName}"` + : `New category for "${dirName}"`, + createdBy: 'system', + }; +} diff --git a/src/types/index.ts b/src/types/index.ts index 1003100..51e371b 100644 --- a/src/types/index.ts +++ b/src/types/index.ts @@ -460,6 +460,121 @@ export interface AssistantWorkspaceFiles { claude?: string; } +export interface AssistantWorkspaceFilesV2 extends AssistantWorkspaceFiles { + dailyMemories?: string[]; + rootReadme?: string; + rootPath?: string; + rootDir?: string; +} + +// ========================================== +// Workspace Inspect Types +// ========================================== + +export interface WorkspaceInspectResult { + exists: boolean; + isDirectory: boolean; + readable: boolean; + writable: boolean; + hasAssistantData: boolean; + workspaceStatus: 'empty' | 'normal_directory' | 'existing_workspace' | 'partial_workspace' | 'invalid'; + summary?: { + onboardingComplete: boolean; + lastCheckInDate: string | null; + fileCount: number; + }; +} + +// ========================================== +// Workspace Config Types +// ========================================== + +export interface AssistantWorkspaceConfig { + workspaceType: string; + organizationStyle: 'project' | 'time' | 'topic' | 'mixed'; + captureDefault: string; + archivePolicy: { + completedTaskArchiveAfterDays: number; + closedProjectArchive: boolean; + dailyMemoryRetentionDays: number; + }; + ignore: string[]; + index: { + maxFileSizeKB: number; + chunkSize: number; + chunkOverlap: number; + maxDepth: number; + includeExtensions: string[]; + }; +} + +// ========================================== +// Taxonomy Types +// ========================================== + +export interface TaxonomyCategory { + id: string; + label: string; + paths: string[]; + role: string; + confidence: number; + source: 'user' | 'learned' | 'default'; + description: string; + createdBy: string; +} + +export interface TaxonomyFile { + version: number; + categories: TaxonomyCategory[]; + evolutionRules: { + allowAutoCreateCategory: boolean; + allowAutoArchive: boolean; + requireConfirmationForMoves: boolean; + }; +} + +// ========================================== +// Workspace Index Types +// ========================================== + +export interface ManifestEntry { + noteId: string; + path: string; + title: string; + aliases: string[]; + tags: string[]; + headings: string[]; + mtime: number; + size: number; + hash: string; + summary: string; + categoryIds: string[]; +} + +export interface ChunkEntry { + chunkId: string; + noteId: string; + path: string; + heading: string; + text: string; + startLine: number; + endLine: number; +} + +export interface HotsetFile { + pinned: string[]; + frequent: Array<{ path: string; count: number; lastAccessed: number }>; + lastUpdated: number; +} + +export interface SearchResult { + path: string; + heading: string; + snippet: string; + score: number; + source: 'title' | 'heading' | 'tag' | 'content'; +} + // ========================================== // Reference Image Types (for image generation) // ==========================================