first commit
This commit is contained in:
commit
f1ea5d241b
16 files changed
+1715
No files matched your search
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"name": "autodl-remote-local-marketplace",
|
||||
"interface": {
|
||||
"displayName": "AutoDL Remote Marketplace"
|
||||
},
|
||||
"plugins": [
|
||||
{
|
||||
"name": "autodl-remote",
|
||||
"source": {
|
||||
"source": "local",
|
||||
"path": "./plugins/autodl-remote"
|
||||
},
|
||||
"policy": {
|
||||
"installation": "AVAILABLE",
|
||||
"authentication": "ON_USE"
|
||||
},
|
||||
"category": "Developer Tools"
|
||||
}
|
||||
]
|
||||
}
|
||||
+48
@@ -0,0 +1,48 @@
|
||||
.DS_Store
|
||||
.codex/
|
||||
.autodl-remote/
|
||||
.autodl-remote.conf
|
||||
*.log
|
||||
*.tmp
|
||||
|
||||
# Python and Node scratch
|
||||
__pycache__/
|
||||
*.pyc
|
||||
node_modules/
|
||||
.venv/
|
||||
venv/
|
||||
env/
|
||||
|
||||
# ML artifacts that should not be published
|
||||
data/
|
||||
dataset/
|
||||
datasets/
|
||||
checkpoints/
|
||||
checkpoint/
|
||||
ckpt/
|
||||
weights/
|
||||
pretrained/
|
||||
models/
|
||||
outputs/
|
||||
output/
|
||||
results/
|
||||
result/
|
||||
runs/
|
||||
wandb/
|
||||
mlruns/
|
||||
*.pt
|
||||
*.pth
|
||||
*.ckpt
|
||||
*.safetensors
|
||||
*.bin
|
||||
*.onnx
|
||||
*.gguf
|
||||
*.h5
|
||||
|
||||
# Local secrets
|
||||
.env
|
||||
.env.*
|
||||
*.pem
|
||||
*.key
|
||||
id_rsa
|
||||
id_ed25519
|
||||
@@ -0,0 +1,13 @@
|
||||
# Changelog
|
||||
|
||||
## 0.5.0
|
||||
|
||||
Initial public package.
|
||||
|
||||
- Added Bash-only `autodl-remote` CLI.
|
||||
- Added Codex plugin metadata and skill instructions.
|
||||
- Added repo-local marketplace manifest.
|
||||
- Added SSH account management.
|
||||
- Added explicit `put`, `get`, `sync-up`, and `sync-down` file operations.
|
||||
- Added remote `exec`, `tree`, `ls`, `cat`, `tail`, and `shell` commands.
|
||||
- Removed earlier source-of-truth modes, manifest tracking, and automatic sync behavior.
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026 Zhuanz
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,272 @@
|
||||
<div align="right">
|
||||
中文 | English planned
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="./docs/images/autodl-remote-hero.svg" alt="AutoDL Remote 工作原理" width="100%">
|
||||
<h1>AutoDL Remote</h1>
|
||||
<h3>让 Codex 留在本地,让 GPU 在远端跑</h3>
|
||||
<p><em>一个面向 Codex App 的极简 SSH 插件:账号管理、远端命令、显式上传和下载。</em></p>
|
||||
<p>
|
||||
<img src="https://img.shields.io/badge/Codex-Plugin-111827?style=flat&logo=openai&logoColor=white" alt="Codex Plugin">
|
||||
<img src="https://img.shields.io/badge/Bash-only-4EAA25?style=flat&logo=gnubash&logoColor=white" alt="Bash only">
|
||||
<img src="https://img.shields.io/badge/SSH-Remote-blue?style=flat" alt="SSH Remote">
|
||||
<img src="https://img.shields.io/badge/AutoDL-GPU-brightgreen?style=flat" alt="AutoDL GPU">
|
||||
<img src="https://img.shields.io/badge/License-MIT-yellow?style=flat" alt="MIT License">
|
||||
</p>
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
## 🎯 项目介绍
|
||||
|
||||
  AutoDL Remote 的目标很直接:**Codex 在本地写代码,远端 GPU 机器负责运行代码;需要什么文件,就显式上传或下载什么文件。**
|
||||
|
||||
  做 LLM、深度学习、论文复现实验时,模型权重、数据集、checkpoint 往往都在 AutoDL 这类远端 GPU 环境里。本地电脑不适合下载大模型,直接在远端安装和登录 Codex 又容易遇到代理、浏览器登录、环境配置等问题。
|
||||
|
||||
  这个插件选择更简单的路线:远端不需要安装 daemon,不需要运行服务,只要能 SSH 登录即可。Codex 仍然在本地工作,本地保留代码修改痕迹和 Git diff;远端负责执行训练、推理和查看日志。
|
||||
|
||||
> **一句话理解:** AutoDL Remote 不是同步框架,而是给 Codex 用的一组 SSH 原语。
|
||||
|
||||
## ✨ 核心特性
|
||||
|
||||
| 能力 | 说明 |
|
||||
| --- | --- |
|
||||
| 🔐 SSH 账号管理 | 支持保存多个远端账号,项目里只引用账号名,不写密码 |
|
||||
| 📁 项目绑定 | 当前本地目录绑定到一个具体远端目录,例如 `/root/autodl-tmp/my-project` |
|
||||
| 🖥️ 远端命令 | 用 `exec` 在远端运行训练、推理、环境检查等命令 |
|
||||
| ⬆️ 显式上传 | 用 `put` 或 `sync-up` 把本地改过的文件送到远端 |
|
||||
| ⬇️ 显式下载 | 用 `get` 或 `sync-down` 只拉回需要查看或提交的结果 |
|
||||
| 📜 日志查看 | 用 `tail` 查看远端训练日志,不必下载大文件 |
|
||||
| 🧩 Codex 友好 | 插件不替你判断同步策略,把决策权留给用户和 Codex |
|
||||
|
||||
## 🧠 工作原理
|
||||
|
||||
<div align="center">
|
||||
<img src="./docs/images/autodl-remote-workflow.svg" alt="AutoDL Remote 典型工作流" width="100%">
|
||||
</div>
|
||||
|
||||
  典型流程是:先查看远端项目结构,再把需要编辑的脚本拉到本地;Codex 在本地修改后,显式上传对应文件;训练或推理在远端执行;最后只查看日志或拉回必要结果。
|
||||
|
||||
<div align="center">
|
||||
<img src="./docs/images/autodl-remote-boundary.svg" alt="AutoDL Remote 本地远端边界" width="100%">
|
||||
</div>
|
||||
|
||||
  这个边界很重要:**本地负责代码和修改痕迹,远端负责大模型、数据集、checkpoint 和 GPU 任务。** 插件不会默认下载模型权重,也不会默认把整个远端目录同步回来。
|
||||
|
||||
## 📚 目录导航
|
||||
|
||||
| 文档 | 内容 |
|
||||
| --- | --- |
|
||||
| [插件目录](./plugins/autodl-remote) | Codex App 插件源码 |
|
||||
| [插件说明](./plugins/autodl-remote/README.md) | CLI 命令和插件内部说明 |
|
||||
| [设计说明](./docs/design.md) | 为什么做成极简 SSH 工具层 |
|
||||
| [故障排查](./docs/troubleshooting.md) | 安装、SSH、路径、权限等问题 |
|
||||
| [更新记录](./CHANGELOG.md) | 版本变化 |
|
||||
|
||||
## 🚀 快速开始
|
||||
|
||||
### 1. 在 Codex App 里安装
|
||||
|
||||
1. Clone 这个仓库。
|
||||
2. 打开 Codex App。
|
||||
3. 进入 `插件` 页面。
|
||||
4. 打开插件来源下拉框,选择 `+ 添加更多`。
|
||||
5. 选择这个仓库的根目录。
|
||||
|
||||
应该选择:
|
||||
|
||||
```text
|
||||
/path/to/autodl-remote
|
||||
```
|
||||
|
||||
不要选择:
|
||||
|
||||
```text
|
||||
/path/to/autodl-remote/plugins/autodl-remote
|
||||
```
|
||||
|
||||
Codex App 会在仓库根目录下读取:
|
||||
|
||||
```text
|
||||
.agents/plugins/marketplace.json
|
||||
```
|
||||
|
||||
添加 marketplace 后,在插件列表里选择 `AutoDL Remote` 并安装/启用。
|
||||
|
||||
### 2. 安装 CLI
|
||||
|
||||
```bash
|
||||
./scripts/install-cli.sh
|
||||
```
|
||||
|
||||
默认会创建软链接:
|
||||
|
||||
```text
|
||||
/opt/homebrew/bin/autodl-remote
|
||||
```
|
||||
|
||||
也可以指定安装目录:
|
||||
|
||||
```bash
|
||||
./scripts/install-cli.sh ~/.local/bin
|
||||
```
|
||||
|
||||
### 3. 添加远端账号
|
||||
|
||||
密码登录:
|
||||
|
||||
```bash
|
||||
autodl-remote account add autodl-gpu \
|
||||
--target root@connect.example.com \
|
||||
--port 2222 \
|
||||
--auth prompt \
|
||||
--default-remote /root/autodl-tmp
|
||||
```
|
||||
|
||||
SSH key 登录:
|
||||
|
||||
```bash
|
||||
autodl-remote account add gpu-key \
|
||||
--target root@host \
|
||||
--port 22 \
|
||||
--key ~/.ssh/id_rsa \
|
||||
--auth ssh-key
|
||||
```
|
||||
|
||||
查看和选择账号:
|
||||
|
||||
```bash
|
||||
autodl-remote account list
|
||||
autodl-remote account use autodl-gpu
|
||||
autodl-remote account test autodl-gpu
|
||||
```
|
||||
|
||||
### 4. 绑定当前项目
|
||||
|
||||
```bash
|
||||
cd /path/to/local/project
|
||||
autodl-remote bind --account autodl-gpu --remote /root/autodl-tmp/my-project
|
||||
autodl-remote doctor
|
||||
```
|
||||
|
||||
项目配置会写入当前目录:
|
||||
|
||||
```bash
|
||||
ACCOUNT="autodl-gpu"
|
||||
REMOTE_ROOT="/root/autodl-tmp/my-project"
|
||||
```
|
||||
|
||||
密码不会写入项目配置。
|
||||
|
||||
## 🛠️ 常用命令
|
||||
|
||||
| 命令 | 用途 |
|
||||
| --- | --- |
|
||||
| `autodl-remote doctor` | 检查当前项目绑定、SSH 连通性、远端目录 |
|
||||
| `autodl-remote tree . --depth 2` | 查看远端目录结构 |
|
||||
| `autodl-remote ls .` | 查看远端目录 |
|
||||
| `autodl-remote cat -- train.py` | 查看远端文件内容 |
|
||||
| `autodl-remote get train.py train.py` | 把远端文件拉到本地 |
|
||||
| `autodl-remote put train.py train.py` | 把本地文件上传到远端 |
|
||||
| `autodl-remote sync-up ./src src` | 上传本地目录 |
|
||||
| `autodl-remote sync-down outputs outputs` | 下载远端目录 |
|
||||
| `autodl-remote exec -- python train.py` | 在远端执行命令 |
|
||||
| `autodl-remote exec --detach --name train -- python train.py` | 远端后台运行长任务 |
|
||||
| `autodl-remote tail -- .autodl-remote/logs/train.log` | 查看远端日志 |
|
||||
|
||||
## 🧪 典型场景
|
||||
|
||||
### 远端已有项目
|
||||
|
||||
```bash
|
||||
autodl-remote tree . --depth 2
|
||||
autodl-remote cat -- train.py
|
||||
autodl-remote get train.py train.py
|
||||
|
||||
# Codex 在本地修改 train.py
|
||||
autodl-remote put train.py train.py
|
||||
autodl-remote exec -- python train.py
|
||||
```
|
||||
|
||||
### 本地已有项目
|
||||
|
||||
```bash
|
||||
autodl-remote bind --account autodl-gpu --remote /root/autodl-tmp/my-project
|
||||
autodl-remote sync-up ./src src
|
||||
autodl-remote put train.py train.py
|
||||
autodl-remote exec -- python train.py
|
||||
```
|
||||
|
||||
### 远端训练任务
|
||||
|
||||
```bash
|
||||
autodl-remote exec --detach --name train -- python train.py
|
||||
autodl-remote tail -- .autodl-remote/logs/train.log
|
||||
autodl-remote cat -- outputs/metrics.json
|
||||
autodl-remote get outputs/metrics.json outputs/metrics.json
|
||||
```
|
||||
|
||||
## ✅ 它不做什么
|
||||
|
||||
AutoDL Remote 保持克制,不做这些事:
|
||||
|
||||
- 不判断本地或远端谁是 source of truth;
|
||||
- 不维护 manifest;
|
||||
- 不自动拉取完整远端项目;
|
||||
- 不在每次运行前自动推送整个本地项目;
|
||||
- 不规定你的项目结构;
|
||||
- 不默认下载模型权重、数据集、checkpoint 或大型输出。
|
||||
|
||||
这些决策交给用户和 Codex,因为不同项目的代码组织方式差异很大。
|
||||
|
||||
## 🔐 安全与隐私
|
||||
|
||||
- 账号配置保存在 `~/.autodl-remote/accounts/`。
|
||||
- 项目只保存账号名和远端目录。
|
||||
- 密码不会写进项目文件。
|
||||
- 可选使用 macOS Keychain 保存密码:
|
||||
|
||||
```bash
|
||||
autodl-remote account password-save autodl-gpu
|
||||
autodl-remote account password-delete autodl-gpu
|
||||
```
|
||||
|
||||
网络流量只会发往你配置的 SSH 主机。远端命令和文件传输都由本地 CLI 显式触发。
|
||||
|
||||
## 📦 仓库结构
|
||||
|
||||
```text
|
||||
.
|
||||
├── .agents/plugins/marketplace.json
|
||||
├── plugins/autodl-remote/
|
||||
│ ├── .codex-plugin/plugin.json
|
||||
│ ├── bin/autodl-remote
|
||||
│ ├── config/example.conf
|
||||
│ ├── skills/autodl-remote/SKILL.md
|
||||
│ └── README.md
|
||||
├── docs/
|
||||
│ ├── design.md
|
||||
│ ├── troubleshooting.md
|
||||
│ └── images/
|
||||
├── scripts/install-cli.sh
|
||||
├── CHANGELOG.md
|
||||
├── .gitignore
|
||||
└── LICENSE
|
||||
```
|
||||
|
||||
## 🤝 贡献
|
||||
|
||||
欢迎提交 Issue 或 Pull Request。这个项目会优先保持轻量,不会加入复杂的自动同步策略。如果你想贡献新能力,建议先说明它是否仍然符合三个核心原语:
|
||||
|
||||
- SSH 账号管理;
|
||||
- 远端命令执行;
|
||||
- 显式上传和下载。
|
||||
|
||||
## 📜 License
|
||||
|
||||
MIT
|
||||
|
||||
<div align="center">
|
||||
<p>如果这个项目对你有帮助,欢迎给一个 Star。</p>
|
||||
</div>
|
||||
@@ -0,0 +1,29 @@
|
||||
# Design
|
||||
|
||||
AutoDL Remote is intentionally small.
|
||||
|
||||
It provides:
|
||||
|
||||
- SSH account management,
|
||||
- project-to-remote-directory binding,
|
||||
- remote command execution,
|
||||
- explicit file upload and download.
|
||||
|
||||
It avoids:
|
||||
|
||||
- source-of-truth decisions,
|
||||
- automatic bidirectional sync,
|
||||
- manifest tracking,
|
||||
- project skeleton inference,
|
||||
- remote daemons,
|
||||
- remote Codex installation.
|
||||
|
||||
The intended workflow is:
|
||||
|
||||
1. Inspect remote state.
|
||||
2. Pull only files needed for editing, or edit existing local files.
|
||||
3. Upload selected local changes.
|
||||
4. Run commands on the remote host.
|
||||
5. Inspect logs and small results remotely.
|
||||
|
||||
This keeps Codex local while letting remote GPU hosts run heavy jobs.
|
||||
@@ -0,0 +1,62 @@
|
||||
<svg width="1280" height="520" viewBox="0 0 1280 520" fill="none" xmlns="http://www.w3.org/2000/svg">
|
||||
<defs>
|
||||
<linearGradient id="bg" x1="0" y1="0" x2="1280" y2="520" gradientUnits="userSpaceOnUse">
|
||||
<stop stop-color="#FAFAFA"/>
|
||||
<stop offset="1" stop-color="#F8FAFC"/>
|
||||
</linearGradient>
|
||||
<filter id="shadow" x="-20%" y="-20%" width="140%" height="140%">
|
||||
<feDropShadow dx="0" dy="12" stdDeviation="16" flood-color="#0F172A" flood-opacity="0.10"/>
|
||||
</filter>
|
||||
<marker id="arrowBlue" markerWidth="12" markerHeight="12" refX="9" refY="6" orient="auto">
|
||||
<path d="M2 2L10 6L2 10V2Z" fill="#2563EB"/>
|
||||
</marker>
|
||||
<marker id="arrowGreen" markerWidth="12" markerHeight="12" refX="9" refY="6" orient="auto">
|
||||
<path d="M2 2L10 6L2 10V2Z" fill="#16A34A"/>
|
||||
</marker>
|
||||
</defs>
|
||||
|
||||
<rect width="1280" height="520" rx="34" fill="url(#bg)"/>
|
||||
<text x="640" y="72" text-anchor="middle" fill="#0F172A" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="34" font-weight="800">本地和远端的边界</text>
|
||||
<text x="640" y="107" text-anchor="middle" fill="#64748B" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="19">代码痕迹留在本地,大文件和 GPU 任务留在远端,中间只传你明确选择的路径</text>
|
||||
|
||||
<g filter="url(#shadow)">
|
||||
<rect x="90" y="150" width="402" height="270" rx="28" fill="#EFF6FF" stroke="#93C5FD" stroke-width="2"/>
|
||||
<text x="291" y="195" text-anchor="middle" fill="#1E3A8A" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="25" font-weight="800">本地项目</text>
|
||||
<rect x="132" y="224" width="318" height="38" rx="12" fill="#FFFFFF" stroke="#BFDBFE"/>
|
||||
<text x="154" y="249" fill="#1D4ED8" font-family="SFMono-Regular,Consolas,monospace" font-size="16">src/ train.py notebooks/</text>
|
||||
<rect x="132" y="278" width="318" height="38" rx="12" fill="#FFFFFF" stroke="#BFDBFE"/>
|
||||
<text x="154" y="303" fill="#1D4ED8" font-family="SFMono-Regular,Consolas,monospace" font-size="16">.git diff / commits</text>
|
||||
<rect x="132" y="332" width="318" height="38" rx="12" fill="#FFFFFF" stroke="#BFDBFE"/>
|
||||
<text x="154" y="357" fill="#1D4ED8" font-family="SFMono-Regular,Consolas,monospace" font-size="16">.autodl-remote.conf</text>
|
||||
</g>
|
||||
|
||||
<g filter="url(#shadow)">
|
||||
<rect x="788" y="150" width="402" height="270" rx="28" fill="#ECFDF5" stroke="#86EFAC" stroke-width="2"/>
|
||||
<text x="989" y="195" text-anchor="middle" fill="#14532D" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="25" font-weight="800">远端项目</text>
|
||||
<rect x="830" y="224" width="318" height="38" rx="12" fill="#FFFFFF" stroke="#BBF7D0"/>
|
||||
<text x="852" y="249" fill="#15803D" font-family="SFMono-Regular,Consolas,monospace" font-size="16">/root/autodl-tmp/project</text>
|
||||
<rect x="830" y="278" width="318" height="38" rx="12" fill="#FFFFFF" stroke="#BBF7D0"/>
|
||||
<text x="852" y="303" fill="#15803D" font-family="SFMono-Regular,Consolas,monospace" font-size="16">models/ data/ checkpoints/</text>
|
||||
<rect x="830" y="332" width="318" height="38" rx="12" fill="#FFFFFF" stroke="#BBF7D0"/>
|
||||
<text x="852" y="357" fill="#15803D" font-family="SFMono-Regular,Consolas,monospace" font-size="16">logs/ outputs/ metrics</text>
|
||||
</g>
|
||||
|
||||
<g>
|
||||
<rect x="558" y="176" width="164" height="168" rx="26" fill="#111827"/>
|
||||
<text x="640" y="222" text-anchor="middle" fill="#FFFFFF" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="24" font-weight="800">SSH</text>
|
||||
<text x="640" y="258" text-anchor="middle" fill="#CBD5E1" font-family="SFMono-Regular,Consolas,monospace" font-size="15">put / get</text>
|
||||
<text x="640" y="286" text-anchor="middle" fill="#CBD5E1" font-family="SFMono-Regular,Consolas,monospace" font-size="15">exec / tail</text>
|
||||
<text x="640" y="318" text-anchor="middle" fill="#FDE68A" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="14">不自动同步</text>
|
||||
</g>
|
||||
|
||||
<path d="M500 230H548" stroke="#2563EB" stroke-width="5" stroke-linecap="round" marker-end="url(#arrowBlue)"/>
|
||||
<path d="M732 230H780" stroke="#2563EB" stroke-width="5" stroke-linecap="round" marker-end="url(#arrowBlue)"/>
|
||||
<text x="640" y="145" text-anchor="middle" fill="#1D4ED8" font-family="SFMono-Regular,Consolas,monospace" font-size="16" font-weight="700">上传你决定的代码文件</text>
|
||||
|
||||
<path d="M780 312H732" stroke="#16A34A" stroke-width="5" stroke-linecap="round" marker-end="url(#arrowGreen)"/>
|
||||
<path d="M548 312H500" stroke="#16A34A" stroke-width="5" stroke-linecap="round" marker-end="url(#arrowGreen)"/>
|
||||
<text x="640" y="392" text-anchor="middle" fill="#15803D" font-family="SFMono-Regular,Consolas,monospace" font-size="16" font-weight="700">只拉回需要查看或提交的结果</text>
|
||||
|
||||
<rect x="288" y="452" width="704" height="42" rx="21" fill="#FFFFFF" stroke="#CBD5E1"/>
|
||||
<text x="640" y="479" text-anchor="middle" fill="#334155" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="17">模型权重、数据集、checkpoint 默认不回传,除非你明确 get 或 sync-down</text>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 5.1 KiB |
@@ -0,0 +1,79 @@
|
||||
<svg width="1280" height="520" viewBox="0 0 1280 520" fill="none" xmlns="http://www.w3.org/2000/svg">
|
||||
<defs>
|
||||
<linearGradient id="bg" x1="0" y1="0" x2="1280" y2="520" gradientUnits="userSpaceOnUse">
|
||||
<stop stop-color="#F8FAFC"/>
|
||||
<stop offset="0.52" stop-color="#EFF6FF"/>
|
||||
<stop offset="1" stop-color="#F7FEE7"/>
|
||||
</linearGradient>
|
||||
<linearGradient id="local" x1="92" y1="96" x2="372" y2="356" gradientUnits="userSpaceOnUse">
|
||||
<stop stop-color="#DBEAFE"/>
|
||||
<stop offset="1" stop-color="#BFDBFE"/>
|
||||
</linearGradient>
|
||||
<linearGradient id="bridge" x1="492" y1="130" x2="788" y2="390" gradientUnits="userSpaceOnUse">
|
||||
<stop stop-color="#FDE68A"/>
|
||||
<stop offset="1" stop-color="#FBBF24"/>
|
||||
</linearGradient>
|
||||
<linearGradient id="remote" x1="908" y1="96" x2="1188" y2="356" gradientUnits="userSpaceOnUse">
|
||||
<stop stop-color="#DCFCE7"/>
|
||||
<stop offset="1" stop-color="#86EFAC"/>
|
||||
</linearGradient>
|
||||
<filter id="shadow" x="-20%" y="-20%" width="140%" height="140%">
|
||||
<feDropShadow dx="0" dy="14" stdDeviation="18" flood-color="#0F172A" flood-opacity="0.12"/>
|
||||
</filter>
|
||||
<marker id="arrow" markerWidth="12" markerHeight="12" refX="9" refY="6" orient="auto">
|
||||
<path d="M2 2L10 6L2 10V2Z" fill="#2563EB"/>
|
||||
</marker>
|
||||
<marker id="arrow2" markerWidth="12" markerHeight="12" refX="9" refY="6" orient="auto">
|
||||
<path d="M2 2L10 6L2 10V2Z" fill="#16A34A"/>
|
||||
</marker>
|
||||
</defs>
|
||||
|
||||
<rect width="1280" height="520" rx="36" fill="url(#bg)"/>
|
||||
<rect x="38" y="38" width="1204" height="444" rx="28" stroke="#CBD5E1" stroke-width="2" stroke-dasharray="10 10"/>
|
||||
|
||||
<text x="640" y="74" text-anchor="middle" fill="#0F172A" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="34" font-weight="800">AutoDL Remote</text>
|
||||
<text x="640" y="110" text-anchor="middle" fill="#475569" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="20">本地 Codex 写代码,远端 GPU 跑任务,SSH 负责连接</text>
|
||||
|
||||
<g filter="url(#shadow)">
|
||||
<rect x="90" y="152" width="286" height="212" rx="24" fill="url(#local)" stroke="#93C5FD" stroke-width="2"/>
|
||||
<rect x="120" y="184" width="226" height="116" rx="14" fill="#FFFFFF" stroke="#60A5FA"/>
|
||||
<rect x="140" y="206" width="110" height="12" rx="6" fill="#2563EB"/>
|
||||
<rect x="140" y="232" width="168" height="10" rx="5" fill="#93C5FD"/>
|
||||
<rect x="140" y="256" width="136" height="10" rx="5" fill="#93C5FD"/>
|
||||
<text x="233" y="333" text-anchor="middle" fill="#1E3A8A" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="24" font-weight="800">Local Codex</text>
|
||||
<text x="233" y="356" text-anchor="middle" fill="#1D4ED8" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="15">代码修改与 Git diff 留在本地</text>
|
||||
</g>
|
||||
|
||||
<g filter="url(#shadow)">
|
||||
<rect x="492" y="170" width="296" height="176" rx="26" fill="url(#bridge)" stroke="#F59E0B" stroke-width="2"/>
|
||||
<rect x="548" y="215" width="184" height="56" rx="28" fill="#111827"/>
|
||||
<text x="640" y="252" text-anchor="middle" fill="#FFFFFF" font-family="SFMono-Regular,Consolas,monospace" font-size="24" font-weight="800">ssh / scp</text>
|
||||
<text x="640" y="310" text-anchor="middle" fill="#78350F" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="23" font-weight="800">显式工具层</text>
|
||||
<text x="640" y="334" text-anchor="middle" fill="#92400E" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="15">exec · put · get · tree · tail</text>
|
||||
</g>
|
||||
|
||||
<g filter="url(#shadow)">
|
||||
<rect x="904" y="152" width="286" height="212" rx="24" fill="url(#remote)" stroke="#4ADE80" stroke-width="2"/>
|
||||
<rect x="935" y="184" width="224" height="116" rx="14" fill="#FFFFFF" stroke="#22C55E"/>
|
||||
<rect x="957" y="209" width="180" height="18" rx="9" fill="#16A34A"/>
|
||||
<rect x="957" y="243" width="70" height="36" rx="9" fill="#BBF7D0"/>
|
||||
<rect x="1043" y="243" width="94" height="36" rx="9" fill="#86EFAC"/>
|
||||
<text x="1047" y="333" text-anchor="middle" fill="#14532D" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="24" font-weight="800">AutoDL GPU</text>
|
||||
<text x="1047" y="356" text-anchor="middle" fill="#15803D" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="15">模型、数据、训练产物留在远端</text>
|
||||
</g>
|
||||
|
||||
<path d="M390 226C423 210 454 210 484 226" stroke="#2563EB" stroke-width="5" stroke-linecap="round" marker-end="url(#arrow)"/>
|
||||
<text x="438" y="202" text-anchor="middle" fill="#1D4ED8" font-family="SFMono-Regular,Consolas,monospace" font-size="17" font-weight="700">put / sync-up</text>
|
||||
|
||||
<path d="M890 292C856 308 823 308 794 292" stroke="#16A34A" stroke-width="5" stroke-linecap="round" marker-end="url(#arrow2)"/>
|
||||
<text x="842" y="333" text-anchor="middle" fill="#15803D" font-family="SFMono-Regular,Consolas,monospace" font-size="17" font-weight="700">get / cat / tail</text>
|
||||
|
||||
<path d="M790 226C823 210 856 210 890 226" stroke="#2563EB" stroke-width="5" stroke-linecap="round" marker-end="url(#arrow)"/>
|
||||
<text x="840" y="202" text-anchor="middle" fill="#1D4ED8" font-family="SFMono-Regular,Consolas,monospace" font-size="17" font-weight="700">exec</text>
|
||||
|
||||
<path d="M484 292C454 308 423 308 390 292" stroke="#16A34A" stroke-width="5" stroke-linecap="round" marker-end="url(#arrow2)"/>
|
||||
<text x="438" y="333" text-anchor="middle" fill="#15803D" font-family="SFMono-Regular,Consolas,monospace" font-size="17" font-weight="700">result when needed</text>
|
||||
|
||||
<rect x="253" y="421" width="774" height="42" rx="21" fill="#FFFFFF" stroke="#CBD5E1"/>
|
||||
<text x="640" y="448" text-anchor="middle" fill="#334155" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="17">插件不判断谁覆盖谁,只给 Codex 提供可组合的 SSH 原语</text>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 5.8 KiB |
@@ -0,0 +1,68 @@
|
||||
<svg width="1280" height="500" viewBox="0 0 1280 500" fill="none" xmlns="http://www.w3.org/2000/svg">
|
||||
<defs>
|
||||
<linearGradient id="bg" x1="0" y1="0" x2="1280" y2="500" gradientUnits="userSpaceOnUse">
|
||||
<stop stop-color="#FFFFFF"/>
|
||||
<stop offset="1" stop-color="#F1F5F9"/>
|
||||
</linearGradient>
|
||||
<filter id="shadow" x="-20%" y="-20%" width="140%" height="140%">
|
||||
<feDropShadow dx="0" dy="10" stdDeviation="14" flood-color="#0F172A" flood-opacity="0.10"/>
|
||||
</filter>
|
||||
<marker id="arrow" markerWidth="12" markerHeight="12" refX="9" refY="6" orient="auto">
|
||||
<path d="M2 2L10 6L2 10V2Z" fill="#334155"/>
|
||||
</marker>
|
||||
</defs>
|
||||
|
||||
<rect width="1280" height="500" rx="34" fill="url(#bg)"/>
|
||||
<text x="640" y="70" text-anchor="middle" fill="#0F172A" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="34" font-weight="800">一次典型远端开发循环</text>
|
||||
<text x="640" y="104" text-anchor="middle" fill="#64748B" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="19">先看清远端,再本地改代码,按需上传,远端运行,最后只取需要的结果</text>
|
||||
|
||||
<path d="M206 250H1074" stroke="#CBD5E1" stroke-width="6" stroke-linecap="round"/>
|
||||
|
||||
<g filter="url(#shadow)">
|
||||
<rect x="80" y="160" width="180" height="180" rx="22" fill="#EFF6FF" stroke="#93C5FD" stroke-width="2"/>
|
||||
<circle cx="170" cy="208" r="30" fill="#2563EB"/>
|
||||
<text x="170" y="218" text-anchor="middle" fill="#FFFFFF" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="28" font-weight="800">1</text>
|
||||
<text x="170" y="270" text-anchor="middle" fill="#1E3A8A" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="22" font-weight="800">查看远端</text>
|
||||
<text x="170" y="300" text-anchor="middle" fill="#2563EB" font-family="SFMono-Regular,Consolas,monospace" font-size="15">tree / cat</text>
|
||||
</g>
|
||||
|
||||
<g filter="url(#shadow)">
|
||||
<rect x="320" y="160" width="180" height="180" rx="22" fill="#F8FAFC" stroke="#CBD5E1" stroke-width="2"/>
|
||||
<circle cx="410" cy="208" r="30" fill="#475569"/>
|
||||
<text x="410" y="218" text-anchor="middle" fill="#FFFFFF" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="28" font-weight="800">2</text>
|
||||
<text x="410" y="270" text-anchor="middle" fill="#0F172A" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="22" font-weight="800">本地修改</text>
|
||||
<text x="410" y="300" text-anchor="middle" fill="#475569" font-family="SFMono-Regular,Consolas,monospace" font-size="15">edit + git diff</text>
|
||||
</g>
|
||||
|
||||
<g filter="url(#shadow)">
|
||||
<rect x="560" y="160" width="180" height="180" rx="22" fill="#FEF3C7" stroke="#FBBF24" stroke-width="2"/>
|
||||
<circle cx="650" cy="208" r="30" fill="#D97706"/>
|
||||
<text x="650" y="218" text-anchor="middle" fill="#FFFFFF" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="28" font-weight="800">3</text>
|
||||
<text x="650" y="270" text-anchor="middle" fill="#78350F" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="22" font-weight="800">按需上传</text>
|
||||
<text x="650" y="300" text-anchor="middle" fill="#B45309" font-family="SFMono-Regular,Consolas,monospace" font-size="15">put / sync-up</text>
|
||||
</g>
|
||||
|
||||
<g filter="url(#shadow)">
|
||||
<rect x="800" y="160" width="180" height="180" rx="22" fill="#ECFDF5" stroke="#86EFAC" stroke-width="2"/>
|
||||
<circle cx="890" cy="208" r="30" fill="#16A34A"/>
|
||||
<text x="890" y="218" text-anchor="middle" fill="#FFFFFF" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="28" font-weight="800">4</text>
|
||||
<text x="890" y="270" text-anchor="middle" fill="#14532D" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="22" font-weight="800">远端运行</text>
|
||||
<text x="890" y="300" text-anchor="middle" fill="#15803D" font-family="SFMono-Regular,Consolas,monospace" font-size="15">exec --detach</text>
|
||||
</g>
|
||||
|
||||
<g filter="url(#shadow)">
|
||||
<rect x="1040" y="160" width="180" height="180" rx="22" fill="#F5F3FF" stroke="#C4B5FD" stroke-width="2"/>
|
||||
<circle cx="1130" cy="208" r="30" fill="#7C3AED"/>
|
||||
<text x="1130" y="218" text-anchor="middle" fill="#FFFFFF" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="28" font-weight="800">5</text>
|
||||
<text x="1130" y="270" text-anchor="middle" fill="#4C1D95" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="22" font-weight="800">查看结果</text>
|
||||
<text x="1130" y="300" text-anchor="middle" fill="#6D28D9" font-family="SFMono-Regular,Consolas,monospace" font-size="15">tail / get</text>
|
||||
</g>
|
||||
|
||||
<path d="M260 250H315" stroke="#334155" stroke-width="4" marker-end="url(#arrow)"/>
|
||||
<path d="M500 250H555" stroke="#334155" stroke-width="4" marker-end="url(#arrow)"/>
|
||||
<path d="M740 250H795" stroke="#334155" stroke-width="4" marker-end="url(#arrow)"/>
|
||||
<path d="M980 250H1035" stroke="#334155" stroke-width="4" marker-end="url(#arrow)"/>
|
||||
|
||||
<rect x="312" y="393" width="656" height="42" rx="21" fill="#FFFFFF" stroke="#CBD5E1"/>
|
||||
<text x="640" y="420" text-anchor="middle" fill="#334155" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="17">每一步都是显式命令,适合让 Codex 根据当前任务自主组合</text>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 5.3 KiB |
@@ -0,0 +1,75 @@
|
||||
# Troubleshooting
|
||||
|
||||
## The Codex App Does Not Show The Plugin
|
||||
|
||||
Make sure you add the repository root as a local marketplace:
|
||||
|
||||
```text
|
||||
/path/to/autodl-remote
|
||||
```
|
||||
|
||||
The root must contain:
|
||||
|
||||
```text
|
||||
.agents/plugins/marketplace.json
|
||||
```
|
||||
|
||||
Do not add:
|
||||
|
||||
```text
|
||||
/path/to/autodl-remote/plugins/autodl-remote
|
||||
```
|
||||
|
||||
After adding the marketplace, fully quit and restart Codex App.
|
||||
|
||||
## The CLI Says `.autodl-remote.conf` Is Missing
|
||||
|
||||
Bind the current local project to a remote project directory:
|
||||
|
||||
```bash
|
||||
autodl-remote bind --account autodl-gpu --remote /root/autodl-tmp/my-project
|
||||
```
|
||||
|
||||
The config should look like:
|
||||
|
||||
```bash
|
||||
ACCOUNT="autodl-gpu"
|
||||
REMOTE_ROOT="/root/autodl-tmp/my-project"
|
||||
```
|
||||
|
||||
## Old Config Files
|
||||
|
||||
Older experimental versions used fields such as `PROJECT_MODE`, `SSH_TARGET`, and `AUTO_PUSH_BEFORE_RUN`.
|
||||
|
||||
The current version does not read that format. Rename the old config and bind again:
|
||||
|
||||
```bash
|
||||
mv .autodl-remote.conf .autodl-remote.conf.old
|
||||
autodl-remote bind --account autodl-gpu --remote /root/autodl-tmp/my-project
|
||||
```
|
||||
|
||||
## Password Login Hangs Or Fails
|
||||
|
||||
For password-based SSH, use prompt auth:
|
||||
|
||||
```bash
|
||||
autodl-remote account add autodl-gpu --target root@host --port 2222 --auth prompt
|
||||
```
|
||||
|
||||
If you need non-interactive password login on macOS, store the password in Keychain:
|
||||
|
||||
```bash
|
||||
autodl-remote account password-save autodl-gpu
|
||||
```
|
||||
|
||||
## Avoid Pulling Large Files
|
||||
|
||||
Do not run broad `get` or `sync-down` commands on model, checkpoint, dataset, or output directories unless you really want those files locally.
|
||||
|
||||
Prefer remote inspection:
|
||||
|
||||
```bash
|
||||
autodl-remote tree . --depth 2
|
||||
autodl-remote tail -- logs/train.log
|
||||
autodl-remote cat -- results/metrics.json
|
||||
```
|
||||
@@ -0,0 +1,42 @@
|
||||
{
|
||||
"name": "autodl-remote",
|
||||
"version": "0.5.0",
|
||||
"description": "Bash-only SSH account, command, upload, and download tool layer for AutoDL.",
|
||||
"author": {
|
||||
"name": "Zhuanz",
|
||||
"email": "maintainer@example.com",
|
||||
"url": "https://github.com/Zhuanz/autodl-remote"
|
||||
},
|
||||
"homepage": "https://github.com/Zhuanz/autodl-remote",
|
||||
"repository": "https://github.com/Zhuanz/autodl-remote",
|
||||
"license": "MIT",
|
||||
"keywords": [
|
||||
"autodl",
|
||||
"ssh",
|
||||
"remote-execution",
|
||||
"rsync",
|
||||
"codex"
|
||||
],
|
||||
"skills": "./skills/",
|
||||
"interface": {
|
||||
"displayName": "AutoDL Remote",
|
||||
"shortDescription": "Control AutoDL over SSH from local Codex.",
|
||||
"longDescription": "Provides SSH account management, remote command execution, and explicit file upload/download primitives. Codex decides what to inspect, push, pull, and run.",
|
||||
"developerName": "Zhuanz",
|
||||
"category": "Developer Tools",
|
||||
"capabilities": [
|
||||
"Read",
|
||||
"Write",
|
||||
"Interactive"
|
||||
],
|
||||
"websiteURL": "https://github.com/Zhuanz/autodl-remote",
|
||||
"privacyPolicyURL": "https://github.com/Zhuanz/autodl-remote#privacy",
|
||||
"termsOfServiceURL": "https://github.com/Zhuanz/autodl-remote#license",
|
||||
"defaultPrompt": [
|
||||
"Use AutoDL Remote to inspect my remote project.",
|
||||
"Upload this edited file to AutoDL and run training.",
|
||||
"Tail the latest AutoDL training log."
|
||||
],
|
||||
"brandColor": "#2563EB"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,154 @@
|
||||
# AutoDL Remote
|
||||
|
||||
AutoDL Remote is a thin SSH tool layer for Codex.
|
||||
|
||||
It does not decide whether local code or remote code is the source of truth. Codex and the user decide what to inspect, upload, download, and run.
|
||||
|
||||
The remote server does not need Codex, Python, a proxy, an OpenAI login, or a daemon.
|
||||
|
||||
## Model
|
||||
|
||||
The plugin has three primitives:
|
||||
|
||||
- SSH account management
|
||||
- Remote command execution
|
||||
- File upload/download
|
||||
|
||||
Global account config lives in:
|
||||
|
||||
```bash
|
||||
~/.autodl-remote/accounts/
|
||||
```
|
||||
|
||||
Project binding lives in the project root:
|
||||
|
||||
```bash
|
||||
.autodl-remote.conf
|
||||
```
|
||||
|
||||
The project file only stores:
|
||||
|
||||
```bash
|
||||
ACCOUNT="autodl-gpu"
|
||||
REMOTE_ROOT="/root/autodl-tmp/my-project"
|
||||
```
|
||||
|
||||
Passwords are never written to project files.
|
||||
|
||||
## Quick Start
|
||||
|
||||
Create or choose an account:
|
||||
|
||||
```bash
|
||||
autodl-remote account add autodl-gpu \
|
||||
--target root@connect.example.com \
|
||||
--port 2222 \
|
||||
--auth prompt \
|
||||
--default-remote /root/autodl-tmp
|
||||
|
||||
autodl-remote account use autodl-gpu
|
||||
autodl-remote account list
|
||||
```
|
||||
|
||||
Bind the current local directory to a concrete remote directory:
|
||||
|
||||
```bash
|
||||
autodl-remote bind --remote /root/autodl-tmp/my-project
|
||||
```
|
||||
|
||||
Inspect and run:
|
||||
|
||||
```bash
|
||||
autodl-remote doctor
|
||||
autodl-remote tree . --depth 2
|
||||
autodl-remote exec -- pwd
|
||||
autodl-remote exec -- python train.py
|
||||
autodl-remote exec --detach --name train -- python train.py
|
||||
autodl-remote tail -- .autodl-remote/logs/train.log
|
||||
```
|
||||
|
||||
Move files only when needed:
|
||||
|
||||
```bash
|
||||
autodl-remote put ./train.py train.py
|
||||
autodl-remote get outputs/result.csv ./outputs/result.csv
|
||||
autodl-remote sync-up ./src src
|
||||
autodl-remote sync-down logs/train.log ./logs/train.log
|
||||
```
|
||||
|
||||
## Accounts
|
||||
|
||||
List accounts:
|
||||
|
||||
```bash
|
||||
autodl-remote account list
|
||||
```
|
||||
|
||||
Show an account without exposing passwords:
|
||||
|
||||
```bash
|
||||
autodl-remote account show autodl-gpu
|
||||
```
|
||||
|
||||
Set the default account:
|
||||
|
||||
```bash
|
||||
autodl-remote account use autodl-gpu
|
||||
```
|
||||
|
||||
Password login is the default for AutoDL:
|
||||
|
||||
```bash
|
||||
autodl-remote account add autodl-gpu --target root@host --port 2222 --auth prompt
|
||||
```
|
||||
|
||||
SSH key login:
|
||||
|
||||
```bash
|
||||
autodl-remote account add gpu-key --target root@host --port 22 --key ~/.ssh/id_rsa --auth ssh-key
|
||||
```
|
||||
|
||||
Optional macOS Keychain password storage:
|
||||
|
||||
```bash
|
||||
autodl-remote account password-save autodl-gpu
|
||||
```
|
||||
|
||||
When an account uses `AUTH="keychain"`, the CLI retrieves the password from macOS Keychain and drives password prompts with `expect`.
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
autodl-remote account add <name> [--target root@host] [--port 22] [--key path] [--auth prompt|keychain|ssh-key] [--default-remote path]
|
||||
autodl-remote account list
|
||||
autodl-remote account show <name>
|
||||
autodl-remote account use <name>
|
||||
autodl-remote account test [name]
|
||||
autodl-remote account password-save <name>
|
||||
autodl-remote account password-delete <name>
|
||||
|
||||
autodl-remote bind [--account name] --remote /remote/project/root
|
||||
autodl-remote doctor
|
||||
|
||||
autodl-remote exec [--detach] [--cwd path] [--name name] -- <command>
|
||||
autodl-remote put <local-path> [remote-path]
|
||||
autodl-remote get <remote-path> [local-path]
|
||||
autodl-remote sync-up <local-path> [remote-path]
|
||||
autodl-remote sync-down <remote-path> [local-path]
|
||||
autodl-remote tree [remote-path] [--depth 3] [--limit 500]
|
||||
autodl-remote ls [remote-path]
|
||||
autodl-remote cat [--lines 200] -- <remote-path>
|
||||
autodl-remote tail [--lines 120] [--follow] -- <remote-path>
|
||||
autodl-remote shell
|
||||
autodl-remote history
|
||||
```
|
||||
|
||||
## Design Rules
|
||||
|
||||
- No `local-main` or `remote-main`.
|
||||
- No manifest.
|
||||
- No automatic code ownership decision.
|
||||
- No automatic pull before edit.
|
||||
- No automatic push before run.
|
||||
- Use `put/get/sync-up/sync-down` explicitly.
|
||||
- Keep large models, datasets, checkpoints, and training outputs on AutoDL unless the user explicitly pulls them.
|
||||
Executable
+748
@@ -0,0 +1,748 @@
|
||||
#!/usr/bin/env bash
|
||||
set -u
|
||||
|
||||
PROGRAM="autodl-remote"
|
||||
APP_DIR="$HOME/.autodl-remote"
|
||||
ACCOUNTS_DIR="$APP_DIR/accounts"
|
||||
DEFAULT_ACCOUNT_FILE="$APP_DIR/default-account"
|
||||
CONFIG_FILE=".autodl-remote.conf"
|
||||
HISTORY_DIR=".autodl-remote"
|
||||
KEYCHAIN_SERVICE="autodl-remote"
|
||||
|
||||
die() {
|
||||
printf '%s: %s\n' "$PROGRAM" "$*" >&2
|
||||
exit 2
|
||||
}
|
||||
|
||||
info() {
|
||||
printf '%s\n' "$*"
|
||||
}
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
usage: autodl-remote <command> [options]
|
||||
|
||||
account:
|
||||
account add <name> [--target root@host | --host host --user root] [--port 22] [--key path] [--auth prompt|keychain|ssh-key] [--default-remote path]
|
||||
account list
|
||||
account show <name>
|
||||
account use <name>
|
||||
account test [name]
|
||||
account password-save <name>
|
||||
account password-delete <name>
|
||||
|
||||
project:
|
||||
bind [--account name] --remote /remote/project/root
|
||||
doctor
|
||||
|
||||
remote:
|
||||
exec [--detach] [--cwd path] [--name name] [--allow-dangerous] -- <command>
|
||||
put <local-path> [remote-path]
|
||||
get <remote-path> [local-path]
|
||||
sync-up <local-path> [remote-path]
|
||||
sync-down <remote-path> [local-path]
|
||||
tree [remote-path] [--depth 3] [--limit 500]
|
||||
ls [remote-path]
|
||||
cat [--lines 200] -- <remote-path>
|
||||
tail [--lines 120] [--follow] -- <remote-path>
|
||||
shell
|
||||
history [--lines 20]
|
||||
|
||||
EOF
|
||||
}
|
||||
|
||||
shell_quote() {
|
||||
printf '%q' "$1"
|
||||
}
|
||||
|
||||
strip_quotes() {
|
||||
local value="$1"
|
||||
value="${value#"${value%%[![:space:]]*}"}"
|
||||
value="${value%"${value##*[![:space:]]}"}"
|
||||
value="${value%%#*}"
|
||||
value="${value%"${value##*[![:space:]]}"}"
|
||||
case "$value" in
|
||||
\"*\") value="${value#\"}"; value="${value%\"}" ;;
|
||||
\'*\') value="${value#\'}"; value="${value%\'}" ;;
|
||||
esac
|
||||
printf '%s' "$value"
|
||||
}
|
||||
|
||||
json_escape() {
|
||||
sed 's/\\/\\\\/g; s/"/\\"/g' <<EOF
|
||||
$1
|
||||
EOF
|
||||
}
|
||||
|
||||
valid_name() {
|
||||
case "$1" in
|
||||
""|*/*|*..*|*[\ \ ]*) return 1 ;;
|
||||
esac
|
||||
return 0
|
||||
}
|
||||
|
||||
ensure_app_dirs() {
|
||||
mkdir -p "$ACCOUNTS_DIR" "$APP_DIR"
|
||||
}
|
||||
|
||||
account_path() {
|
||||
local name="$1"
|
||||
valid_name "$name" || die "Invalid account name: $name"
|
||||
ensure_app_dirs
|
||||
printf '%s/%s.conf\n' "$ACCOUNTS_DIR" "$name"
|
||||
}
|
||||
|
||||
config_get_from() {
|
||||
local file="$1" key="$2"
|
||||
awk -F= -v key="$key" '
|
||||
$0 ~ "^[[:space:]]*" key "[[:space:]]*=" {
|
||||
sub(/^[^=]*=/, "", $0)
|
||||
print $0
|
||||
exit
|
||||
}
|
||||
' "$file" | while IFS= read -r line; do strip_quotes "$line"; done
|
||||
}
|
||||
|
||||
find_project_config() {
|
||||
if [ -n "${AUTODL_REMOTE_CONFIG:-}" ]; then
|
||||
[ -f "$AUTODL_REMOTE_CONFIG" ] || die "AUTODL_REMOTE_CONFIG points to a missing file: $AUTODL_REMOTE_CONFIG"
|
||||
printf '%s\n' "$AUTODL_REMOTE_CONFIG"
|
||||
return 0
|
||||
fi
|
||||
|
||||
local dir
|
||||
dir="$(pwd)"
|
||||
while :; do
|
||||
if [ -f "$dir/$CONFIG_FILE" ]; then
|
||||
printf '%s\n' "$dir/$CONFIG_FILE"
|
||||
return 0
|
||||
fi
|
||||
[ "$dir" = "/" ] && break
|
||||
dir="$(dirname "$dir")"
|
||||
done
|
||||
die "No $CONFIG_FILE found. Run 'autodl-remote bind --account <name> --remote <path>'."
|
||||
}
|
||||
|
||||
validate_remote_root() {
|
||||
local root="${1%/}"
|
||||
[ "${root#/}" != "$root" ] || die "remote path must be absolute: $root"
|
||||
[ "$root" != "/" ] || die "remote path is too broad: /"
|
||||
case "$root" in
|
||||
*"/../"*|*/..) die "remote path cannot contain '..': $root" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
write_history() {
|
||||
local action="$1" detail="$2" file escaped
|
||||
mkdir -p "$PROJECT_ROOT/$HISTORY_DIR"
|
||||
file="$PROJECT_ROOT/$HISTORY_DIR/history.jsonl"
|
||||
escaped="$(json_escape "$detail")"
|
||||
printf '{"ts":"%s","action":"%s","detail":"%s"}\n' "$(date +%Y-%m-%dT%H:%M:%S%z)" "$action" "$escaped" >> "$file"
|
||||
}
|
||||
|
||||
load_account() {
|
||||
local name="$1" path
|
||||
valid_name "$name" || die "Invalid account name: $name"
|
||||
path="$(account_path "$name")"
|
||||
[ -f "$path" ] || die "Account not found: $name"
|
||||
|
||||
ACCOUNT_NAME="$name"
|
||||
ACCOUNT_FILE="$path"
|
||||
ACCOUNT_HOST="$(config_get_from "$path" HOST)"
|
||||
ACCOUNT_USER="$(config_get_from "$path" USER)"
|
||||
ACCOUNT_TARGET="$(config_get_from "$path" TARGET)"
|
||||
ACCOUNT_PORT="$(config_get_from "$path" PORT)"
|
||||
ACCOUNT_KEY="$(config_get_from "$path" KEY)"
|
||||
ACCOUNT_AUTH="$(config_get_from "$path" AUTH)"
|
||||
ACCOUNT_DEFAULT_REMOTE="$(config_get_from "$path" DEFAULT_REMOTE)"
|
||||
|
||||
if [ -z "$ACCOUNT_TARGET" ]; then
|
||||
[ -n "$ACCOUNT_HOST" ] || die "Account $name is missing HOST/TARGET."
|
||||
[ -n "$ACCOUNT_USER" ] || die "Account $name is missing USER/TARGET."
|
||||
ACCOUNT_TARGET="$ACCOUNT_USER@$ACCOUNT_HOST"
|
||||
fi
|
||||
ACCOUNT_PORT="${ACCOUNT_PORT:-22}"
|
||||
ACCOUNT_KEY="${ACCOUNT_KEY:-}"
|
||||
ACCOUNT_AUTH="${ACCOUNT_AUTH:-prompt}"
|
||||
ACCOUNT_DEFAULT_REMOTE="${ACCOUNT_DEFAULT_REMOTE:-}"
|
||||
|
||||
case "$ACCOUNT_AUTH" in
|
||||
prompt|keychain|ssh-key) ;;
|
||||
*) die "Account $name has invalid AUTH: $ACCOUNT_AUTH" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
default_account() {
|
||||
if [ -f "$DEFAULT_ACCOUNT_FILE" ]; then
|
||||
sed -n '1p' "$DEFAULT_ACCOUNT_FILE"
|
||||
return 0
|
||||
fi
|
||||
return 1
|
||||
}
|
||||
|
||||
load_project() {
|
||||
CONFIG_PATH="$(find_project_config)"
|
||||
PROJECT_ROOT="$(cd "$(dirname "$CONFIG_PATH")" && pwd)"
|
||||
PROJECT_ACCOUNT="$(config_get_from "$CONFIG_PATH" ACCOUNT)"
|
||||
REMOTE_ROOT="$(config_get_from "$CONFIG_PATH" REMOTE_ROOT)"
|
||||
|
||||
[ -n "$PROJECT_ACCOUNT" ] || die "$CONFIG_FILE is missing ACCOUNT."
|
||||
load_account "$PROJECT_ACCOUNT"
|
||||
|
||||
[ -n "$REMOTE_ROOT" ] || REMOTE_ROOT="$ACCOUNT_DEFAULT_REMOTE"
|
||||
[ -n "$REMOTE_ROOT" ] || die "$CONFIG_FILE is missing REMOTE_ROOT."
|
||||
REMOTE_ROOT="${REMOTE_ROOT%/}"
|
||||
validate_remote_root "$REMOTE_ROOT"
|
||||
}
|
||||
|
||||
ssh_common_args() {
|
||||
SSH_ARGS=(-p "$ACCOUNT_PORT" -o ServerAliveInterval=30 -o ServerAliveCountMax=3 -o ControlMaster=auto -o ControlPersist=600 -o "ControlPath=$APP_DIR/ssh-%C")
|
||||
mkdir -p "$APP_DIR"
|
||||
[ -n "$ACCOUNT_KEY" ] && SSH_ARGS+=(-i "$ACCOUNT_KEY")
|
||||
if [ "$ACCOUNT_AUTH" != "keychain" ] && [ "$ACCOUNT_AUTH" != "prompt" ]; then
|
||||
SSH_ARGS+=(-o BatchMode=yes)
|
||||
fi
|
||||
}
|
||||
|
||||
rsync_ssh_command() {
|
||||
local cmd
|
||||
cmd="ssh -p $(shell_quote "$ACCOUNT_PORT") -o ServerAliveInterval=30 -o ServerAliveCountMax=3 -o ControlMaster=auto -o ControlPersist=600 -o ControlPath=$(shell_quote "$APP_DIR/ssh-%C")"
|
||||
[ -n "$ACCOUNT_KEY" ] && cmd="$cmd -i $(shell_quote "$ACCOUNT_KEY")"
|
||||
if [ "$ACCOUNT_AUTH" != "keychain" ] && [ "$ACCOUNT_AUTH" != "prompt" ]; then
|
||||
cmd="$cmd -o BatchMode=yes"
|
||||
fi
|
||||
printf '%s\n' "$cmd"
|
||||
}
|
||||
|
||||
keychain_password() {
|
||||
command -v security >/dev/null 2>&1 || die "macOS security command is not available."
|
||||
security find-generic-password -a "$ACCOUNT_NAME" -s "$KEYCHAIN_SERVICE" -w 2>/dev/null
|
||||
}
|
||||
|
||||
expect_with_password() {
|
||||
command -v expect >/dev/null 2>&1 || die "Account uses keychain auth, but expect is not available."
|
||||
local password="$1"
|
||||
shift
|
||||
AUTODL_REMOTE_PASSWORD="$password" expect -c '
|
||||
set timeout -1
|
||||
set password $env(AUTODL_REMOTE_PASSWORD)
|
||||
spawn {*}$argv
|
||||
expect {
|
||||
-re "(?i)are you sure you want to continue connecting" {
|
||||
send "yes\r"
|
||||
exp_continue
|
||||
}
|
||||
-re "(?i)password:" {
|
||||
send "$password\r"
|
||||
exp_continue
|
||||
}
|
||||
eof
|
||||
}
|
||||
catch wait result
|
||||
exit [lindex $result 3]
|
||||
' "$@"
|
||||
}
|
||||
|
||||
run_transport() {
|
||||
if [ "$ACCOUNT_AUTH" = "keychain" ]; then
|
||||
local password
|
||||
password="$(keychain_password)" || die "No Keychain password saved for account: $ACCOUNT_NAME"
|
||||
expect_with_password "$password" "$@"
|
||||
else
|
||||
"$@"
|
||||
fi
|
||||
}
|
||||
|
||||
ssh_run() {
|
||||
local command="$1"
|
||||
ssh_common_args
|
||||
run_transport ssh "${SSH_ARGS[@]}" "$ACCOUNT_TARGET" "$command"
|
||||
}
|
||||
|
||||
ssh_capture() {
|
||||
local command="$1"
|
||||
ssh_common_args
|
||||
run_transport ssh "${SSH_ARGS[@]}" "$ACCOUNT_TARGET" "$command"
|
||||
}
|
||||
|
||||
remote_join() {
|
||||
local path="${1:-.}"
|
||||
if [ -z "$path" ] || [ "$path" = "." ]; then
|
||||
printf '%s\n' "$REMOTE_ROOT"
|
||||
return 0
|
||||
fi
|
||||
case "$path" in
|
||||
/*)
|
||||
validate_remote_root "$path"
|
||||
printf '%s\n' "${path%/}"
|
||||
;;
|
||||
../*|*/../*|..)
|
||||
die "path cannot escape remote root: $path"
|
||||
;;
|
||||
*)
|
||||
printf '%s/%s\n' "$REMOTE_ROOT" "${path#./}"
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
remote_rel_for_local() {
|
||||
local path="$1"
|
||||
case "$path" in
|
||||
/*)
|
||||
if [ "${path#"$PROJECT_ROOT"/}" != "$path" ]; then
|
||||
printf '%s\n' "${path#"$PROJECT_ROOT"/}"
|
||||
else
|
||||
basename "$path"
|
||||
fi
|
||||
;;
|
||||
*)
|
||||
printf '%s\n' "${path#./}"
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
local_join() {
|
||||
local path="$1"
|
||||
case "$path" in
|
||||
/*) printf '%s\n' "$path" ;;
|
||||
*) printf '%s/%s\n' "$PROJECT_ROOT" "${path#./}" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
remote_has_rsync() {
|
||||
ssh_capture "command -v rsync >/dev/null 2>&1" >/dev/null 2>&1
|
||||
}
|
||||
|
||||
ensure_remote_dir() {
|
||||
local dir="$1"
|
||||
ssh_capture "mkdir -p $(shell_quote "$dir")" >/dev/null
|
||||
}
|
||||
|
||||
dangerous_command_check() {
|
||||
local cmd="$1"
|
||||
case "$cmd" in
|
||||
*"rm -rf /"*|*"mkfs"*|*"shutdown"*|*"reboot"*|*"poweroff"*|*"dd if="*)
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
return 0
|
||||
}
|
||||
|
||||
cmd_account_add() {
|
||||
local name="" target="" host="" user="" port="22" key="" auth="prompt" default_remote="" force=false
|
||||
[ $# -gt 0 ] || die "account add requires a name"
|
||||
name="$1"
|
||||
shift
|
||||
valid_name "$name" || die "Invalid account name: $name"
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--target) target="$2"; shift 2 ;;
|
||||
--host) host="$2"; shift 2 ;;
|
||||
--user) user="$2"; shift 2 ;;
|
||||
--port) port="$2"; shift 2 ;;
|
||||
--key) key="$2"; shift 2 ;;
|
||||
--auth) auth="$2"; shift 2 ;;
|
||||
--default-remote) default_remote="$2"; shift 2 ;;
|
||||
--force) force=true; shift ;;
|
||||
*) die "unknown account add option: $1" ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [ -z "$target" ]; then
|
||||
if [ -z "$host" ] && [ -t 0 ]; then
|
||||
printf 'host: '
|
||||
read -r host
|
||||
fi
|
||||
if [ -z "$user" ] && [ -t 0 ]; then
|
||||
printf 'user [root]: '
|
||||
read -r user
|
||||
user="${user:-root}"
|
||||
fi
|
||||
[ -n "$host" ] || die "account add requires --target or --host."
|
||||
[ -n "$user" ] || user="root"
|
||||
target="$user@$host"
|
||||
fi
|
||||
|
||||
case "$auth" in
|
||||
prompt|keychain|ssh-key) ;;
|
||||
*) die "--auth must be prompt, keychain, or ssh-key" ;;
|
||||
esac
|
||||
[ -z "$default_remote" ] || validate_remote_root "$default_remote"
|
||||
[ "$auth" != "ssh-key" ] || [ -n "$key" ] || die "--auth ssh-key requires --key"
|
||||
|
||||
local path
|
||||
path="$(account_path "$name")"
|
||||
[ ! -f "$path" ] || [ "$force" = "true" ] || die "Account already exists: $name. Use --force to overwrite."
|
||||
cat > "$path" <<EOF
|
||||
TARGET="$target"
|
||||
PORT="$port"
|
||||
KEY="$key"
|
||||
AUTH="$auth"
|
||||
DEFAULT_REMOTE="$default_remote"
|
||||
EOF
|
||||
chmod 600 "$path"
|
||||
info "Saved account: $name"
|
||||
}
|
||||
|
||||
cmd_account_list() {
|
||||
ensure_app_dirs
|
||||
local seen=false file name
|
||||
if ls "$ACCOUNTS_DIR"/*.conf >/dev/null 2>&1; then
|
||||
for file in "$ACCOUNTS_DIR"/*.conf; do
|
||||
seen=true
|
||||
name="$(basename "$file" .conf)"
|
||||
if [ "$(default_account 2>/dev/null || true)" = "$name" ]; then
|
||||
printf '* %s\n' "$name"
|
||||
else
|
||||
printf ' %s\n' "$name"
|
||||
fi
|
||||
done
|
||||
fi
|
||||
[ "$seen" = "true" ] || info "No accounts saved."
|
||||
}
|
||||
|
||||
cmd_account_show() {
|
||||
[ $# -gt 0 ] || die "account show requires a name"
|
||||
load_account "$1"
|
||||
info "name=$ACCOUNT_NAME"
|
||||
info "target=$ACCOUNT_TARGET"
|
||||
info "port=$ACCOUNT_PORT"
|
||||
info "key=$ACCOUNT_KEY"
|
||||
info "auth=$ACCOUNT_AUTH"
|
||||
info "default_remote=$ACCOUNT_DEFAULT_REMOTE"
|
||||
}
|
||||
|
||||
cmd_account_use() {
|
||||
[ $# -gt 0 ] || die "account use requires a name"
|
||||
load_account "$1"
|
||||
ensure_app_dirs
|
||||
printf '%s\n' "$ACCOUNT_NAME" > "$DEFAULT_ACCOUNT_FILE"
|
||||
info "Default account: $ACCOUNT_NAME"
|
||||
}
|
||||
|
||||
cmd_account_test() {
|
||||
local name="${1:-}"
|
||||
if [ -z "$name" ]; then
|
||||
name="$(default_account)" || die "No default account. Run 'autodl-remote account use <name>'."
|
||||
fi
|
||||
load_account "$name"
|
||||
info "Testing $ACCOUNT_NAME -> $ACCOUNT_TARGET:$ACCOUNT_PORT"
|
||||
ssh_capture "printf 'host=' && hostname && printf 'pwd=' && pwd && printf 'rsync=' && { command -v rsync || true; }"
|
||||
}
|
||||
|
||||
cmd_account_password_save() {
|
||||
[ $# -gt 0 ] || die "account password-save requires a name"
|
||||
load_account "$1"
|
||||
command -v security >/dev/null 2>&1 || die "macOS security command is not available."
|
||||
local password=""
|
||||
if [ -n "${AUTODL_REMOTE_PASSWORD:-}" ]; then
|
||||
password="$AUTODL_REMOTE_PASSWORD"
|
||||
else
|
||||
[ -t 0 ] || die "Set AUTODL_REMOTE_PASSWORD or run from an interactive terminal."
|
||||
printf 'password for %s: ' "$ACCOUNT_NAME" >&2
|
||||
stty -echo
|
||||
read -r password
|
||||
stty echo
|
||||
printf '\n' >&2
|
||||
fi
|
||||
[ -n "$password" ] || die "Empty password not saved."
|
||||
security add-generic-password -U -a "$ACCOUNT_NAME" -s "$KEYCHAIN_SERVICE" -w "$password" >/dev/null
|
||||
local path
|
||||
path="$(account_path "$ACCOUNT_NAME")"
|
||||
if [ ! -f "$path" ]; then
|
||||
local add_args=("$ACCOUNT_NAME" --target "$ACCOUNT_TARGET" --port "$ACCOUNT_PORT" --auth keychain --default-remote "$ACCOUNT_DEFAULT_REMOTE" --force)
|
||||
[ -n "$ACCOUNT_KEY" ] && add_args+=(--key "$ACCOUNT_KEY")
|
||||
cmd_account_add "${add_args[@]}" >/dev/null
|
||||
else
|
||||
awk -F= 'BEGIN{done=0} $1=="AUTH"{print "AUTH=\"keychain\""; done=1; next} {print} END{if(!done) print "AUTH=\"keychain\""}' "$path" > "$path.tmp"
|
||||
mv "$path.tmp" "$path"
|
||||
chmod 600 "$path"
|
||||
fi
|
||||
info "Saved password in macOS Keychain for account: $ACCOUNT_NAME"
|
||||
}
|
||||
|
||||
cmd_account_password_delete() {
|
||||
[ $# -gt 0 ] || die "account password-delete requires a name"
|
||||
load_account "$1"
|
||||
command -v security >/dev/null 2>&1 || die "macOS security command is not available."
|
||||
security delete-generic-password -a "$ACCOUNT_NAME" -s "$KEYCHAIN_SERVICE" >/dev/null 2>&1 || true
|
||||
info "Deleted Keychain password for account: $ACCOUNT_NAME"
|
||||
}
|
||||
|
||||
cmd_account() {
|
||||
[ $# -gt 0 ] || die "account requires a subcommand"
|
||||
local sub="$1"
|
||||
shift
|
||||
case "$sub" in
|
||||
add) cmd_account_add "$@" ;;
|
||||
list|ls) cmd_account_list "$@" ;;
|
||||
show) cmd_account_show "$@" ;;
|
||||
use) cmd_account_use "$@" ;;
|
||||
test) cmd_account_test "$@" ;;
|
||||
password-save) cmd_account_password_save "$@" ;;
|
||||
password-delete) cmd_account_password_delete "$@" ;;
|
||||
*) die "unknown account command: $sub" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
cmd_bind() {
|
||||
local account="" remote="" force=false
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--account) account="$2"; shift 2 ;;
|
||||
--remote) remote="$2"; shift 2 ;;
|
||||
--force) force=true; shift ;;
|
||||
*) die "unknown bind option: $1" ;;
|
||||
esac
|
||||
done
|
||||
if [ -z "$account" ]; then
|
||||
account="$(default_account)" || die "bind requires --account or a default account."
|
||||
fi
|
||||
load_account "$account"
|
||||
[ -n "$remote" ] || remote="$ACCOUNT_DEFAULT_REMOTE"
|
||||
[ -n "$remote" ] || die "bind requires --remote."
|
||||
remote="${remote%/}"
|
||||
validate_remote_root "$remote"
|
||||
[ ! -f "$CONFIG_FILE" ] || [ "$force" = "true" ] || die "$CONFIG_FILE already exists. Use --force to overwrite."
|
||||
cat > "$CONFIG_FILE" <<EOF
|
||||
ACCOUNT="$ACCOUNT_NAME"
|
||||
REMOTE_ROOT="$remote"
|
||||
EOF
|
||||
mkdir -p "$HISTORY_DIR"
|
||||
info "Bound $(pwd) to $ACCOUNT_NAME:$remote"
|
||||
}
|
||||
|
||||
cmd_doctor() {
|
||||
load_project
|
||||
info "Config: $CONFIG_PATH"
|
||||
info "Local root: $PROJECT_ROOT"
|
||||
info "Account: $ACCOUNT_NAME"
|
||||
info "Target: $ACCOUNT_TARGET"
|
||||
info "Port: $ACCOUNT_PORT"
|
||||
info "Remote root: $REMOTE_ROOT"
|
||||
info "Local ssh: $(command -v ssh || echo MISSING)"
|
||||
info "Local rsync: $(command -v rsync || echo MISSING)"
|
||||
ssh_capture "printf 'host=' && hostname && printf 'pwd=' && pwd && printf 'remote_root=' && { test -d $(shell_quote "$REMOTE_ROOT") && echo exists || echo missing; } && printf 'rsync=' && { command -v rsync || true; } && command -v nvidia-smi >/dev/null 2>&1 && nvidia-smi --query-gpu=name,memory.total --format=csv,noheader || true"
|
||||
}
|
||||
|
||||
cmd_exec() {
|
||||
load_project
|
||||
local detach=false cwd="$REMOTE_ROOT" name="" allow_dangerous=false
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--detach) detach=true; shift ;;
|
||||
--cwd) cwd="$(remote_join "$2")"; shift 2 ;;
|
||||
--name) name="$2"; shift 2 ;;
|
||||
--allow-dangerous) allow_dangerous=true; shift ;;
|
||||
--) shift; break ;;
|
||||
*) break ;;
|
||||
esac
|
||||
done
|
||||
[ $# -gt 0 ] || die "exec requires a command"
|
||||
local cmd="$*"
|
||||
if [ "$allow_dangerous" != "true" ]; then
|
||||
dangerous_command_check "$cmd" || die "Dangerous remote command blocked. Re-run with --allow-dangerous after explicit approval."
|
||||
fi
|
||||
[ -n "$name" ] || name="exec-$(date +%Y%m%d-%H%M%S)"
|
||||
name="$(printf '%s' "$name" | sed 's/[^A-Za-z0-9_.-]/-/g')"
|
||||
local log="$REMOTE_ROOT/.autodl-remote/logs/$name.log"
|
||||
info "log=$log"
|
||||
if [ "$detach" = "true" ]; then
|
||||
ssh_capture "mkdir -p $(shell_quote "$(dirname "$log")") && cd $(shell_quote "$cwd") && nohup bash -lc $(shell_quote "$cmd") > $(shell_quote "$log") 2>&1 < /dev/null & printf 'pid=%s\nlog=%s\n' \"\$!\" $(shell_quote "$log")"
|
||||
write_history exec-detach "$cmd log=$log"
|
||||
else
|
||||
local inner rc
|
||||
inner="set -o pipefail; ( $cmd ) 2>&1 | tee $(shell_quote "$log"); exit \${PIPESTATUS[0]}"
|
||||
ssh_run "mkdir -p $(shell_quote "$(dirname "$log")") && cd $(shell_quote "$cwd") && bash -lc $(shell_quote "$inner")"
|
||||
rc=$?
|
||||
write_history exec "$cmd rc=$rc log=$log"
|
||||
return "$rc"
|
||||
fi
|
||||
}
|
||||
|
||||
cmd_put() {
|
||||
load_project
|
||||
local dry_run=false delete=false
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--dry-run) dry_run=true; shift ;;
|
||||
--delete) delete=true; shift ;;
|
||||
--) shift; break ;;
|
||||
*) break ;;
|
||||
esac
|
||||
done
|
||||
[ $# -ge 1 ] || die "put requires <local-path> [remote-path]"
|
||||
local local_arg="$1" remote_arg="${2:-}" local_path remote_path
|
||||
local_path="$(local_join "$local_arg")"
|
||||
[ -e "$local_path" ] || die "Local path does not exist: $local_arg"
|
||||
if [ -z "$remote_arg" ]; then
|
||||
remote_arg="$(remote_rel_for_local "$local_path")"
|
||||
fi
|
||||
remote_path="$(remote_join "$remote_arg")"
|
||||
ensure_remote_dir "$(dirname "$remote_path")"
|
||||
if command -v rsync >/dev/null 2>&1 && remote_has_rsync; then
|
||||
local args=(-az --human-readable -e "$(rsync_ssh_command)")
|
||||
[ "$dry_run" = "true" ] && args+=(--dry-run)
|
||||
[ "$delete" = "true" ] && args+=(--delete)
|
||||
if [ -d "$local_path" ]; then
|
||||
run_transport rsync "${args[@]}" "$local_path/" "$ACCOUNT_TARGET:$remote_path/"
|
||||
else
|
||||
run_transport rsync "${args[@]}" "$local_path" "$ACCOUNT_TARGET:$remote_path"
|
||||
fi
|
||||
else
|
||||
[ "$dry_run" != "true" ] || die "dry-run requires rsync."
|
||||
ssh_common_args
|
||||
local scp_args=(-P "$ACCOUNT_PORT")
|
||||
[ -n "$ACCOUNT_KEY" ] && scp_args+=(-i "$ACCOUNT_KEY")
|
||||
if [ -d "$local_path" ]; then
|
||||
run_transport scp -r "${scp_args[@]}" "$local_path" "$ACCOUNT_TARGET:$remote_path"
|
||||
else
|
||||
run_transport scp "${scp_args[@]}" "$local_path" "$ACCOUNT_TARGET:$remote_path"
|
||||
fi
|
||||
fi
|
||||
write_history put "$local_arg -> $remote_path"
|
||||
}
|
||||
|
||||
cmd_get() {
|
||||
load_project
|
||||
local dry_run=false
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--dry-run) dry_run=true; shift ;;
|
||||
--) shift; break ;;
|
||||
*) break ;;
|
||||
esac
|
||||
done
|
||||
[ $# -ge 1 ] || die "get requires <remote-path> [local-path]"
|
||||
local remote_arg="$1" local_arg="${2:-}" remote_path local_path kind
|
||||
remote_path="$(remote_join "$remote_arg")"
|
||||
if [ -z "$local_arg" ]; then
|
||||
case "$remote_arg" in
|
||||
/*) local_arg="$(basename "$remote_arg")" ;;
|
||||
*) local_arg="$remote_arg" ;;
|
||||
esac
|
||||
fi
|
||||
local_path="$(local_join "$local_arg")"
|
||||
mkdir -p "$(dirname "$local_path")"
|
||||
kind="$(ssh_capture "if [ -d $(shell_quote "$remote_path") ]; then echo dir; elif [ -f $(shell_quote "$remote_path") ]; then echo file; else echo missing; fi")"
|
||||
[ "$kind" != "missing" ] || die "Remote path does not exist: $remote_arg"
|
||||
if command -v rsync >/dev/null 2>&1 && remote_has_rsync; then
|
||||
local args=(-az --human-readable -e "$(rsync_ssh_command)")
|
||||
[ "$dry_run" = "true" ] && args+=(--dry-run)
|
||||
if [ "$kind" = "dir" ]; then
|
||||
mkdir -p "$local_path"
|
||||
run_transport rsync "${args[@]}" "$ACCOUNT_TARGET:$remote_path/" "$local_path/"
|
||||
else
|
||||
run_transport rsync "${args[@]}" "$ACCOUNT_TARGET:$remote_path" "$local_path"
|
||||
fi
|
||||
else
|
||||
[ "$dry_run" != "true" ] || die "dry-run requires rsync."
|
||||
local scp_args=(-P "$ACCOUNT_PORT")
|
||||
[ -n "$ACCOUNT_KEY" ] && scp_args+=(-i "$ACCOUNT_KEY")
|
||||
if [ "$kind" = "dir" ]; then
|
||||
run_transport scp -r "${scp_args[@]}" "$ACCOUNT_TARGET:$remote_path" "$local_path"
|
||||
else
|
||||
run_transport scp "${scp_args[@]}" "$ACCOUNT_TARGET:$remote_path" "$local_path"
|
||||
fi
|
||||
fi
|
||||
write_history get "$remote_path -> $local_arg"
|
||||
}
|
||||
|
||||
cmd_tree() {
|
||||
load_project
|
||||
local path="." depth=3 limit=500
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--depth) depth="$2"; shift 2 ;;
|
||||
--limit) limit="$2"; shift 2 ;;
|
||||
--) shift; break ;;
|
||||
-*) die "unknown tree option: $1" ;;
|
||||
*) path="$1"; shift ;;
|
||||
esac
|
||||
done
|
||||
local remote_path
|
||||
remote_path="$(remote_join "$path")"
|
||||
ssh_capture "cd $(shell_quote "$remote_path") && { find . -maxdepth $depth | sed 's#^./##' | sort | head -n $limit; true; }"
|
||||
}
|
||||
|
||||
cmd_ls() {
|
||||
load_project
|
||||
[ "${1:-}" = "--" ] && shift
|
||||
local path="${1:-.}"
|
||||
ssh_run "ls -lah $(shell_quote "$(remote_join "$path")")"
|
||||
}
|
||||
|
||||
cmd_cat() {
|
||||
load_project
|
||||
local lines=200
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--lines) lines="$2"; shift 2 ;;
|
||||
--) shift; break ;;
|
||||
*) break ;;
|
||||
esac
|
||||
done
|
||||
[ $# -gt 0 ] || die "cat requires a path"
|
||||
ssh_run "sed -n '1,${lines}p' $(shell_quote "$(remote_join "$1")")"
|
||||
}
|
||||
|
||||
cmd_tail() {
|
||||
load_project
|
||||
local lines=120 follow=false
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--lines) lines="$2"; shift 2 ;;
|
||||
--follow|-f) follow=true; shift ;;
|
||||
--) shift; break ;;
|
||||
*) break ;;
|
||||
esac
|
||||
done
|
||||
[ $# -gt 0 ] || die "tail requires a path"
|
||||
local flag=""
|
||||
[ "$follow" = "true" ] && flag="-f"
|
||||
ssh_run "tail -n $lines $flag $(shell_quote "$(remote_join "$1")")"
|
||||
}
|
||||
|
||||
cmd_shell() {
|
||||
load_project
|
||||
ssh_common_args
|
||||
run_transport ssh -t "${SSH_ARGS[@]}" "$ACCOUNT_TARGET" "cd $(shell_quote "$REMOTE_ROOT") && exec bash -l"
|
||||
}
|
||||
|
||||
cmd_history() {
|
||||
load_project
|
||||
local lines=20 file
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--lines) lines="$2"; shift 2 ;;
|
||||
*) die "unknown history option: $1" ;;
|
||||
esac
|
||||
done
|
||||
file="$PROJECT_ROOT/$HISTORY_DIR/history.jsonl"
|
||||
[ -f "$file" ] || { info "No history yet."; return 0; }
|
||||
tail -n "$lines" "$file"
|
||||
}
|
||||
|
||||
main() {
|
||||
[ $# -gt 0 ] || { usage; exit 0; }
|
||||
local cmd="$1"
|
||||
shift
|
||||
case "$cmd" in
|
||||
-h|--help|help) usage ;;
|
||||
account) cmd_account "$@" ;;
|
||||
bind) cmd_bind "$@" ;;
|
||||
doctor) cmd_doctor "$@" ;;
|
||||
exec) cmd_exec "$@" ;;
|
||||
put|sync-up) cmd_put "$@" ;;
|
||||
get|sync-down) cmd_get "$@" ;;
|
||||
tree) cmd_tree "$@" ;;
|
||||
ls) cmd_ls "$@" ;;
|
||||
cat) cmd_cat "$@" ;;
|
||||
tail) cmd_tail "$@" ;;
|
||||
shell) cmd_shell "$@" ;;
|
||||
history) cmd_history "$@" ;;
|
||||
*) die "unknown command: $cmd" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
main "$@"
|
||||
@@ -0,0 +1,2 @@
|
||||
ACCOUNT="autodl-gpu"
|
||||
REMOTE_ROOT="/root/autodl-tmp/YOUR_PROJECT"
|
||||
@@ -0,0 +1,63 @@
|
||||
---
|
||||
name: autodl-remote
|
||||
description: Use when Codex should control an AutoDL or SSH server from the local machine, run remote commands, and explicitly upload or download selected files.
|
||||
---
|
||||
|
||||
# AutoDL Remote
|
||||
|
||||
Use this skill when the user wants Codex to work locally while controlling AutoDL through SSH.
|
||||
|
||||
## Operating Model
|
||||
|
||||
- The plugin is a thin SSH tool layer.
|
||||
- Do not decide that local or remote is the source of truth globally.
|
||||
- Do not use `local-main`, `remote-main`, manifests, sparse-cache modes, or automatic code ownership rules.
|
||||
- Codex decides the smallest useful action each time: inspect, upload, download, run, or tail logs.
|
||||
- Do not download LLM weights, checkpoints, datasets, or large outputs unless the user explicitly asks.
|
||||
- Prefer reading remote files/logs in place with `cat`, `tail`, `ls`, and `tree`.
|
||||
|
||||
## Setup
|
||||
|
||||
Use an existing account or create one:
|
||||
|
||||
```bash
|
||||
autodl-remote account list
|
||||
autodl-remote account add autodl-gpu --target root@host --port 2222 --auth prompt
|
||||
autodl-remote account use autodl-gpu
|
||||
```
|
||||
|
||||
Bind the current local directory to the concrete remote directory the user wants:
|
||||
|
||||
```bash
|
||||
autodl-remote bind --account autodl-gpu --remote /root/autodl-tmp/my-project
|
||||
```
|
||||
|
||||
The project config should only contain `ACCOUNT` and `REMOTE_ROOT`.
|
||||
|
||||
## Core Commands
|
||||
|
||||
```bash
|
||||
autodl-remote doctor
|
||||
autodl-remote tree . --depth 2
|
||||
autodl-remote ls .
|
||||
autodl-remote cat -- README.md
|
||||
autodl-remote tail -- logs/train.log
|
||||
|
||||
autodl-remote put ./train.py train.py
|
||||
autodl-remote get outputs/result.csv ./outputs/result.csv
|
||||
autodl-remote sync-up ./src src
|
||||
autodl-remote sync-down logs/train.log ./logs/train.log
|
||||
|
||||
autodl-remote exec -- pwd
|
||||
autodl-remote exec -- python train.py
|
||||
autodl-remote exec --detach --name train -- python train.py
|
||||
```
|
||||
|
||||
## Behavior Rules
|
||||
|
||||
- Before editing remote-existing code, inspect with `tree`, `ls`, `cat`, or `get` only the needed files.
|
||||
- Before running remote code that depends on local edits, explicitly upload the edited files or directories with `put` or `sync-up`.
|
||||
- After remote execution, inspect logs remotely first; pull only small result files when useful.
|
||||
- If both local and remote have similar files, do not assume one should overwrite the other. Compare or inspect first, then choose `put` or `get`.
|
||||
- Use `exec --detach` for long training jobs and then `tail` the log path.
|
||||
- Treat broad destructive commands as dangerous; require explicit user approval before using `--allow-dangerous`.
|
||||
Executable
+19
@@ -0,0 +1,19 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
SOURCE="$ROOT_DIR/plugins/autodl-remote/bin/autodl-remote"
|
||||
DEST_DIR="${1:-/opt/homebrew/bin}"
|
||||
DEST="$DEST_DIR/autodl-remote"
|
||||
|
||||
if [ ! -f "$SOURCE" ]; then
|
||||
echo "Missing CLI source: $SOURCE" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
mkdir -p "$DEST_DIR"
|
||||
ln -sf "$SOURCE" "$DEST"
|
||||
chmod +x "$SOURCE"
|
||||
|
||||
echo "Installed autodl-remote -> $DEST"
|
||||
echo "Run: autodl-remote --help"
|
||||
Reference in new issue
Block a user