16 Commits
Author SHA1 Message Date
KellyvvandClaude Opus 4.6 12eff570c8 chore: shell-only build, drop gemma-4-e4b-it-4bit from Resources phase
Default release build is the empty-shell IPA — model files live under
Models/ for local development but are no longer copied into the .app
bundle. Users still download Gemma weights at first launch via the
in-app downloader (ModelScope / hf-mirror / huggingface.co fallback).

Keep PBXFileReference + group children so Models/ stays browsable in
Xcode, but remove the BuildFile entry from PBXResourcesBuildPhase.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 09:30:34 +08:00
KellyvvandClaude Opus 4.6 be94b244d2 chore: remove DEVELOPMENT_TEAM from project.pbxproj
Xcode auto-injected the local team identifier when files were added to
the project. Strip it again so the repo stays signing-agnostic.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 09:25:09 +08:00
KellyvvandClaude Opus 4.6 779be78843 refactor: split MLX service, introduce skill type & runtime budgets
Major restructuring of the agent stack to support content-type skills,
data-driven runtime budgets, and the new translate skill.

LLM/MLX/
- Split MLXLocalLLMService.swift (1500 → 700 lines) into Core/, Gemma4/,
  Installation/ subdirectories
- New BundledModel: LinearBudgetFormula replaces tier tables, model
  parameters fully data-driven via ModelRuntimeProfile
- New RuntimeBudgets: pure functions for text/thinking/multimodal budgets,
  zero hardcoded model IDs
- Drop adjustedTextOutputTokens prepared-subtraction (chunked prefill made
  it obsolete and triggered false-positive truncation on long prompts)

Skills/
- Add SkillType (.device / .content) metadata, parsed from SKILL.md
  frontmatter
- New translate skill (type: content, no tools) with anaphora handling
- Trim translate (4563→1804), device (2357→1680), calendar (2169→1790)
- contacts SKILL.md adds multi-turn disambiguation + cancel guidance,
  semantics owned by body not framework
- SkillRegistry validates allowed-tools against ToolRegistry at load time

Agent/
- ToolCallParser becomes registry-aware: drops tool_call names not in
  ToolRegistry or protocol set (load_skill / list_skills); ToolRegistry
  is now the single white list, validation moves to parser boundary
- Router sticky routing uses semantic user-turn boundary instead of
  fixed suffix(4); correctly handles long agent loops
- AgentEngine system prompt template differentiates device vs content
  skills with explicit invocation rules
- SYSPROMPT.md auto-migration: old single-placeholder template backed up
  to .bak when new dual-placeholder is detected
- PromptBuilder.buildLoadedSkillPrompt receives availableTools list,
  explicitly tells model which tools (or none) are callable

Tools/
- ToolRegistry split into Helpers/Stores/AppPermissions/Handlers/* for
  separation of concerns
- Mark Tools/Handlers/* with explicit allowed argument signatures

UI/
- Add long-press copy contextMenu on user and assistant bubbles
- Enable .textSelection on user bubble

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 09:22:20 +08:00
Kellyvv 8872646858 chore: remove fixed development team from project 2026-04-07 16:29:46 +08:00
Kellyvv 69c4b32248 feat: ship shell app and chat session polish 2026-04-07 11:13:35 +08:00
Kellyvv 0d223ae373 feat: add audio input and stabilize gemma 4 inference 2026-04-07 01:50:29 +08:00
Kellyvv b304b40770 docs: remove title from README header (banner first); chore: clear Team ID 2026-04-06 06:10:12 +08:00
Kellyvv 150964c660 chore: remove development team from tracked project settings 2026-04-06 04:05:40 +08:00
Kellyvv 95553ee653 feat: major agent upgrade — multi-model, calendar/reminders/contacts skills
Agent:
- Rewrite load_skill follow-up: inject skill instructions into system
  block (buildLoadedSkillPrompt) instead of appending as user message
- Add auto tool-call shortcut for single no-arg skills
  (autoToolCallForLoadedSkills) — skips second LLM round
- Add Chinese date parsing (parseBasicChineseDate, detectDateInQuestion)
- Add heuristic argument inference (heuristicArgumentsForTool)
- Add looksLikePromptEcho / looksLikeStructuredIntermediateOutput filters
- Add renderToolResultLocally for zero-inference result formatting
- Add permission management API (AppPermissionKind, permissionStatuses,
  requestPermission)
- Add applyModelSelection() for runtime model switching

PromptBuilder:
- Replace buildFollowUp+buildForcedSkillContinuation with:
  buildLoadedSkillPrompt (skill in system block, cleaner prefill)
  buildToolAnswerPrompt (minimal tool-result answer prompt)
- Add extractSystemBlock / injectIntoSystemBlock helpers

MLXLocalLLMService:
- Add BundledModelOption struct
- Support multiple models (E2B + E4B) via availableModels array
- modelPath now dynamic based on selectedModel

ToolRegistry:
- Add calendar-create-event (EventKit, write-only access)
- Add reminders-create (EventKit, full access, EKAlarm for notifications)
  - ensureWritableReminderCalendar: auto-creates reminder list if needed
- Add contacts-upsert (Contacts, dedup by phone number)
- Fix: CNContacts .limited case treated as .granted
- Fix: reminders EKAlarm required for notification to fire

SkillLoader:
- Add calendar, reminders, contacts default SKILL.md entries

Info.plist:
- Add NSCalendarsWriteOnlyAccessUsageDescription
- Add NSCalendarsUsageDescription
- Add NSRemindersFullAccessUsageDescription
- Add NSRemindersUsageDescription
- Add NSContactsUsageDescription

ConfigurationsView:
- Add model picker (E2B / E4B)
- Add permissions panel (AppPermissionKind list with request support)
2026-04-06 03:49:21 +08:00
Kellyvv d5744708ba chore: remove all MediaPipe/E4B/LiteRT legacy references
- Delete doc/HOW_TO_BUILD_E4B_IOS_APP.md (entirely obsolete)
- Podfile: remove MediaPipeTasksGenAI/C pods (not used, code uses MLX SPM)
- pod install: regenerated Podfile.lock (1 pod: Yams only)
- README.md: remove E4B from hardware requirements
- LLMEngine.swift: clean MediaPipe from comments
- MLXLocalLLMService.swift: replace E4B → E2B in inline comments
- PromptBuilder.swift: replace E4B → E2B in inline comments
- AgentEngine.swift: replace LiteRT-LM comment with MLX
- doc/iphone_agent_permissions_*.md: replace E4B → Gemma 4
2026-04-06 02:24:50 +08:00
Kellyvv 4f113662f4 chore: local dev setup — restore Team ID, fix pbxproj structure
- Restore DEVELOPMENT_TEAM = ZP4QNJ35TH (needed for on-device build)
- Reorder PBXBuildFile / PBXGroup entries for Xcode compatibility
- Fix XCLocalSwiftPackageReference path for Packages/InferenceKit
2026-04-06 02:00:47 +08:00
Kellyvv 7e7f137726 refactor: rename mlx-swift-lm → Packages/InferenceKit 2026-04-06 01:53:57 +08:00
Kellyvv 475bf496e4 feat: vendor mlx-swift-lm as local SPM package (2.7MB, no .git)
Replaces remote XCRemoteSwiftPackageReference with local XCLocalSwiftPackageReference.
The mlx-swift-lm/ directory is now self-contained in this repo — anyone who
clones PhoneClaw gets MLXLMCommon, MLXLLM, MLXVLM without any network dependency.
2026-04-06 01:53:06 +08:00
Kellyvv abc25eb24f chore: open-source cleanup
- Remove personal Team ID (ZP4QNJ35TH) and Bundle ID (com.zxw.*)
  from project.pbxproj; replaced with empty team and com.example.phoneclaw
- Rewrite README.md for open-source (remove local paths, update to MLX architecture)
- Fix doc/HOW_TO_BUILD_E4B_IOS_APP.md: remove /Users/zxw/ hardcoded paths
- Delete LiteRT-LM/ (unused C API reference, never integrated)
- Delete _mediapipe-samples/ (reference only, not part of build)
- Delete _mlx-swift-lm/ (local clone, replaced with remote SPM reference)
- Convert _mlx-swift-lm from XCLocalSwiftPackageReference to remote
  XCRemoteSwiftPackageReference → https://github.com/ml-explore/mlx-swift-lm (branch: main)
- Update .gitignore to reflect removed directories
2026-04-06 01:44:55 +08:00
Kellyvv 0f146d4e8f refactor: flatten structure + remove dead code
- Move PhoneClaw/ contents to project root (no more nesting)
- Delete PhoneClawMac/ (nested SPM package, duplicated iOS code)
- Delete LLM/LocalLLMService.swift (dead MediaPipe implementation)
- Delete llm_inference_engine.h + PhoneClaw-Bridging-Header.h (unused)
- Delete Sources/ (empty dir), phoneclaw.code-workspace, TECH_PATH_ANALYSIS.md, project.yml
- Fix project.pbxproj: update INFOPLIST_FILE, CODE_SIGN_ENTITLEMENTS paths
- Fix project.pbxproj: remove SWIFT_OBJC_BRIDGING_HEADER setting
- Fix project.pbxproj: remove LocalLLMService.swift file reference
- Fix project.pbxproj: PhoneClaw group now uses name= instead of path=
2026-04-06 01:33:10 +08:00
Kellyvv d3f7dd78ca feat: initial commit — PhoneClaw on-device AI Agent
- AgentEngine: multi-round tool-chain (up to 10 rounds)
- MLX GPU inference with Gemma 4 E2B (4-bit quantized)
- VLM multimodal support (image input via <|image|> token)
- File-driven Skill system: SKILL.md (YAML frontmatter + body)
- Built-in Skills: clipboard, device, text
- ToolRegistry: iOS native API wrappers
- PromptBuilder: Gemma 4 chat template with function calling
- SwiftUI chat UI with Skill progress cards + thinking animation
- Memory-safe inference: dynamic history depth, GPU cache clearing
- PhoneClawMac: SPM package for macOS port
2026-04-06 01:26:23 +08:00