add shutdown and remote job helpers

CHANGELOG.md

@@ -1,5 +1,13 @@
 # Changelog
 
+## 0.6.0
+
+- Added `autodl-remote shutdown` with shutdown/poweroff/halt fallback and SSH disconnect-aware success handling.
+- Added detached job registry commands: `job list`, `job status <name>`, and `job tail <name>`.
+- Added `exec --script <local_script>` and `exec --stdin -- <command>` for complex multi-line remote commands.
+- Added `put-run <local-path>... -- <command>` to upload selected local paths before running a remote command.
+- Added `model-dir [--mkdir]` and automatic local `.autodl-remote/CONVENTIONS.md` creation for project-level remote conventions.
+
 ## 0.5.0
 
 Initial public package.

README.md

@@ -38,6 +38,10 @@
 | ⬆️ 显式上传 | 用 `put` 或 `sync-up` 把本地改过的文件送到远端 |
 | ⬇️ 显式下载 | 用 `get` 或 `sync-down` 只拉回需要查看或提交的结果 |
 | 📜 日志查看 | 用 `tail` 查看远端训练日志，不必下载大文件 |
+| 🧾 Job 状态 | 用 `job status` 和 `job tail` 跟踪 `exec --detach --name` 创建的长任务 |
+| 🧠 脚本执行 | 用 `exec --script` 或 `exec --stdin` 避免复杂 quoting |
+| ⏻ 远端关机 | 用 `shutdown` 处理 AutoDL 关机和 SSH 断开返回码 |
+| 📝 项目约定 | 首次使用自动生成 `.autodl-remote/CONVENTIONS.md` 记录远端约定 |
 | 🧩 Codex 友好 | 插件不替你判断同步策略，把决策权留给用户和 Codex |
 
 ## ✅ Requirements
@@ -195,11 +199,14 @@ REMOTE_ROOT="/root/autodl-tmp/my-project"
 
 ```bash
 autodl-remote doctor
+autodl-remote model-dir
 autodl-remote tree . --depth 2
 autodl-remote exec -- pwd
 autodl-remote exec -- nvidia-smi
 ```
 
+`model-dir` 会输出项目默认模型目录；第一次执行项目命令时，本地还会自动生成 `.autodl-remote/CONVENTIONS.md`，用来记录模型目录、数据目录、运行命令等项目约定。
+
 如果这几步正常，再开始上传代码或运行训练。
 
 ## 🧠 工作原理
@@ -226,11 +233,18 @@ autodl-remote exec -- nvidia-smi
 | `autodl-remote cat -- train.py` | 查看远端文件内容 |
 | `autodl-remote get train.py train.py` | 把远端文件拉到本地 |
 | `autodl-remote put train.py train.py` | 把本地文件上传到远端 |
+| `autodl-remote put-run train.py -- python train.py` | 先上传文件，再执行远端命令 |
 | `autodl-remote sync-up ./src src` | 上传本地目录 |
 | `autodl-remote sync-down outputs outputs` | 下载远端目录 |
 | `autodl-remote exec -- python train.py` | 在远端执行命令 |
+| `autodl-remote exec --script scripts/check.sh` | 上传并运行本地脚本，减少 shell quoting 问题 |
+| `cat script.sh \| autodl-remote exec --stdin -- bash` | 从 stdin 上传多行脚本并运行 |
 | `autodl-remote exec --detach --name train -- python train.py` | 远端后台运行长任务 |
-| `autodl-remote tail -- .autodl-remote/logs/train.log` | 查看远端日志 |
+| `autodl-remote job list` | 列出本地记录的远端长任务 |
+| `autodl-remote job status train` | 查看长任务 PID、日志、远端状态和退出码 |
+| `autodl-remote job tail train` | 查看命名长任务日志 |
+| `autodl-remote model-dir --mkdir` | 输出并创建项目模型目录 |
+| `autodl-remote shutdown` | 请求远端关机，并把 SSH 断开视为可能成功 |
 
 ## 📌 Usage Patterns
 
@@ -242,8 +256,21 @@ autodl-remote cat -- train.py
 autodl-remote get train.py train.py
 
 # Codex 在本地修改 train.py
-autodl-remote put train.py train.py
-autodl-remote exec -- python train.py
+autodl-remote put-run train.py -- python train.py
+```
+
+### 复杂脚本或 heredoc
+
+```bash
+autodl-remote exec --script scripts/remote_metrics.sh
+
+cat <<'SH' | autodl-remote exec --stdin -- bash
+set -euo pipefail
+pwd
+python - <<'PY'
+print("remote python ok")
+PY
+SH
 ```
 
 ### 本地已有项目
@@ -251,19 +278,48 @@ autodl-remote exec -- python train.py
 ```bash
 autodl-remote bind --account autodl-gpu --remote /root/autodl-tmp/my-project
 autodl-remote sync-up ./src src
-autodl-remote put train.py train.py
-autodl-remote exec -- python train.py
+autodl-remote put-run train.py -- python train.py
 ```
 
 ### 远端训练任务
 
 ```bash
 autodl-remote exec --detach --name train -- python train.py
-autodl-remote tail -- .autodl-remote/logs/train.log
+autodl-remote job status train
+autodl-remote job tail train
 autodl-remote cat -- outputs/metrics.json
 autodl-remote get outputs/metrics.json outputs/metrics.json
 ```
 
+### 远端关机
+
+```bash
+autodl-remote shutdown
+```
+
+`shutdown` 会先执行 `sync`，再尝试 `shutdown`、`poweroff`、`halt`，最后通过 SSH 是否还能连接来判断关机是否生效。关机导致 SSH 返回 `255` 时，会按“连接被远端关闭，可能已关机”处理。
+
+### 项目模型目录约定
+
+```bash
+autodl-remote model-dir
+autodl-remote model-dir --mkdir
+```
+
+默认规则：
+
+- 如果远端路径在 `/root/autodl-tmp` 下，默认模型目录是 `/root/autodl-tmp/models`。
+- 其他远端路径默认使用 `<REMOTE_ROOT>/models`。
+- 可在 `.autodl-remote.conf` 里设置 `MODEL_DIR="/some/path"` 覆盖。
+
+首次项目命令会在本地生成：
+
+```text
+.autodl-remote/CONVENTIONS.md
+```
+
+这个文件用于记录模型位置、数据位置、日志位置、常用运行命令等项目约定。
+
 ## ✅ What This Plugin Does Not Do
 
 AutoDL Remote 保持克制，不做这些事：

plugins/autodl-remote/.codex-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "autodl-remote",
-  "version": "0.5.0",
-  "description": "Bash-only SSH account, command, upload, and download tool layer for AutoDL.",
+  "version": "0.6.0",
+  "description": "Bash-only SSH account, command, upload, download, job, and shutdown tool layer for AutoDL.",
   "author": {
     "name": "haibarazz",
     "url": "https://github.com/haibarazz/AutoDL-Remote"
@@ -13,6 +13,7 @@
     "autodl",
     "ssh",
     "remote-execution",
+    "remote-jobs",
     "rsync",
     "codex"
   ],
@@ -20,7 +21,7 @@
   "interface": {
     "displayName": "AutoDL Remote",
     "shortDescription": "Control AutoDL over SSH from local Codex.",
-    "longDescription": "Provides SSH account management, remote command execution, and explicit file upload/download primitives. Codex decides what to inspect, push, pull, and run.",
+    "longDescription": "Provides SSH account management, remote command execution, explicit file upload/download primitives, remote job tracking, script execution, and shutdown handling. Codex decides what to inspect, push, pull, and run.",
     "developerName": "haibarazz",
     "category": "Developer Tools",
     "capabilities": [
@@ -34,7 +35,9 @@
     "defaultPrompt": [
       "Use AutoDL Remote to inspect my remote project.",
       "Upload this edited file to AutoDL and run training.",
-      "Tail the latest AutoDL training log."
+      "Tail the latest AutoDL training log.",
+      "Check the status of my detached AutoDL job.",
+      "Shut down the remote AutoDL machine when the run is finished."
     ],
     "brandColor": "#2563EB"
   }

plugins/autodl-remote/README.md

@@ -60,22 +60,38 @@ Inspect and run:
 
 ```bash
 autodl-remote doctor
+autodl-remote model-dir
 autodl-remote tree . --depth 2
 autodl-remote exec -- pwd
 autodl-remote exec -- python train.py
 autodl-remote exec --detach --name train -- python train.py
-autodl-remote tail -- .autodl-remote/logs/train.log
+autodl-remote job status train
+autodl-remote job tail train
 ```
 
 Move files only when needed:
 
 ```bash
 autodl-remote put ./train.py train.py
+autodl-remote put-run ./train.py -- python train.py
 autodl-remote get outputs/result.csv ./outputs/result.csv
 autodl-remote sync-up ./src src
 autodl-remote sync-down logs/train.log ./logs/train.log
 ```
 
+Run complex local scripts without shell quoting problems:
+
+```bash
+autodl-remote exec --script scripts/remote_check.sh
+cat scripts/remote_check.sh | autodl-remote exec --stdin -- bash
+```
+
+Shutdown the remote machine after a run:
+
+```bash
+autodl-remote shutdown
+```
+
 ## Accounts
 
 List accounts:
@@ -129,8 +145,10 @@ autodl-remote account password-delete <name>
 
 autodl-remote bind [--account name] --remote /remote/project/root
 autodl-remote doctor
+autodl-remote model-dir [--mkdir]
 
-autodl-remote exec [--detach] [--cwd path] [--name name] -- <command>
+autodl-remote exec [--detach] [--cwd path] [--name name] [--script local_script] [--stdin] -- <command>
+autodl-remote put-run <local-path>... -- <command>
 autodl-remote put <local-path> [remote-path]
 autodl-remote get <remote-path> [local-path]
 autodl-remote sync-up <local-path> [remote-path]
@@ -139,6 +157,10 @@ autodl-remote tree [remote-path] [--depth 3] [--limit 500]
 autodl-remote ls [remote-path]
 autodl-remote cat [--lines 200] -- <remote-path>
 autodl-remote tail [--lines 120] [--follow] -- <remote-path>
+autodl-remote job list
+autodl-remote job status <name>
+autodl-remote job tail [--lines 120] [--follow] <name>
+autodl-remote shutdown [--wait seconds]
 autodl-remote shell
 autodl-remote history
 ```
@@ -152,3 +174,6 @@ autodl-remote history
 - No automatic push before run.
 - Use `put/get/sync-up/sync-down` explicitly.
 - Keep large models, datasets, checkpoints, and training outputs on AutoDL unless the user explicitly pulls them.
+- Use `exec --script` or `exec --stdin` for multi-line shell/Python snippets.
+- Use `job status` and `job tail` for detached training jobs created with `exec --detach --name`.
+- Use `shutdown` instead of guessing provider-specific poweroff commands.

plugins/autodl-remote/bin/autodl-remote

@@ -34,9 +34,11 @@ account:
 project:
   bind [--account name] --remote /remote/project/root
   doctor
+  model-dir [--mkdir]
 
 remote:
-  exec [--detach] [--cwd path] [--name name] [--allow-dangerous] -- <command>
+  exec [--detach] [--cwd path] [--name name] [--script local_script] [--stdin] [--allow-dangerous] -- <command>
+  put-run <local-path>... -- <command>
   put <local-path> [remote-path]
   get <remote-path> [local-path]
   sync-up <local-path> [remote-path]
@@ -45,6 +47,10 @@ remote:
   ls [remote-path]
   cat [--lines 200] -- <remote-path>
   tail [--lines 120] [--follow] -- <remote-path>
+  job list
+  job status <name>
+  job tail [--lines 120] [--follow] <name>
+  shutdown [--wait seconds]
   shell
   history [--lines 20]
 
@@ -140,6 +146,79 @@ write_history() {
   printf '{"ts":"%s","action":"%s","detail":"%s"}\n' "$(date +%Y-%m-%dT%H:%M:%S%z)" "$action" "$escaped" >> "$file"
 }
 
+sanitize_name() {
+  local name="$1"
+  name="$(printf '%s' "$name" | sed 's/[^A-Za-z0-9_.-]/-/g')"
+  [ -n "$name" ] || name="job-$(date +%Y%m%d-%H%M%S)"
+  printf '%s\n' "$name"
+}
+
+config_quote() {
+  sed 's/\\/\\\\/g; s/"/\\"/g' <<EOF
+$1
+EOF
+}
+
+default_model_dir() {
+  case "$REMOTE_ROOT" in
+    /root/autodl-tmp|/root/autodl-tmp/*) printf '%s\n' "/root/autodl-tmp/models" ;;
+    *) printf '%s\n' "$REMOTE_ROOT/models" ;;
+  esac
+}
+
+ensure_project_conventions() {
+  local dir file
+  dir="$PROJECT_ROOT/$HISTORY_DIR"
+  file="$dir/CONVENTIONS.md"
+  mkdir -p "$dir"
+  [ -f "$file" ] && return 0
+  cat > "$file" <<EOF
+# AutoDL Remote Conventions
+
+This file is created locally by \`autodl-remote\` to record project-level remote workflow notes.
+
+- Account: \`$ACCOUNT_NAME\`
+- Remote root: \`$REMOTE_ROOT\`
+- Model directory: \`$PROJECT_MODEL_DIR\`
+- Large files such as models, datasets, checkpoints, and training outputs should stay remote unless explicitly pulled.
+- Edit code locally, upload selected files with \`put\` or \`sync-up\`, run on the remote GPU, then inspect logs/results remotely first.
+
+## Project Notes
+
+- Add project-specific paths, model locations, dataset locations, and run commands here.
+EOF
+}
+
+jobs_dir() {
+  printf '%s/%s/jobs\n' "$PROJECT_ROOT" "$HISTORY_DIR"
+}
+
+job_path() {
+  local name="$1"
+  valid_name "$name" || die "Invalid job name: $name"
+  printf '%s/%s.conf\n' "$(jobs_dir)" "$name"
+}
+
+write_job() {
+  local name="$1" pid="$2" log="$3" status_path="$4" cwd="$5" command="$6" file
+  mkdir -p "$(jobs_dir)"
+  file="$(job_path "$name")"
+  cat > "$file" <<EOF
+NAME="$(config_quote "$name")"
+PID="$(config_quote "$pid")"
+LOG_PATH="$(config_quote "$log")"
+STATUS_PATH="$(config_quote "$status_path")"
+CWD="$(config_quote "$cwd")"
+STARTED_AT="$(date +%Y-%m-%dT%H:%M:%S%z)"
+COMMAND="$(config_quote "$command")"
+EOF
+}
+
+job_get() {
+  local file="$1" key="$2"
+  config_get_from "$file" "$key"
+}
+
 load_account() {
   local name="$1" path
   valid_name "$name" || die "Invalid account name: $name"
@@ -185,6 +264,7 @@ load_project() {
   PROJECT_ROOT="$(cd "$(dirname "$CONFIG_PATH")" && pwd)"
   PROJECT_ACCOUNT="$(config_get_from "$CONFIG_PATH" ACCOUNT)"
   REMOTE_ROOT="$(config_get_from "$CONFIG_PATH" REMOTE_ROOT)"
+  PROJECT_MODEL_DIR="$(config_get_from "$CONFIG_PATH" MODEL_DIR)"
 
   [ -n "$PROJECT_ACCOUNT" ] || die "$CONFIG_FILE is missing ACCOUNT."
   load_account "$PROJECT_ACCOUNT"
@@ -193,6 +273,14 @@ load_project() {
   [ -n "$REMOTE_ROOT" ] || die "$CONFIG_FILE is missing REMOTE_ROOT."
   REMOTE_ROOT="${REMOTE_ROOT%/}"
   validate_remote_root "$REMOTE_ROOT"
+  [ -n "$PROJECT_MODEL_DIR" ] || PROJECT_MODEL_DIR="$(default_model_dir)"
+  case "$PROJECT_MODEL_DIR" in
+    /*) ;;
+    *) PROJECT_MODEL_DIR="$REMOTE_ROOT/${PROJECT_MODEL_DIR#./}" ;;
+  esac
+  PROJECT_MODEL_DIR="${PROJECT_MODEL_DIR%/}"
+  validate_remote_root "$PROJECT_MODEL_DIR"
+  ensure_project_conventions
 }
 
 ssh_common_args() {
@@ -318,6 +406,16 @@ ensure_remote_dir() {
   ssh_capture "mkdir -p $(shell_quote "$dir")" >/dev/null
 }
 
+upload_local_file() {
+  local local_path="$1" remote_path="$2"
+  [ -f "$local_path" ] || die "Local script does not exist or is not a file: $local_path"
+  ensure_remote_dir "$(dirname "$remote_path")"
+  ssh_common_args
+  local scp_args=(-P "$ACCOUNT_PORT")
+  [ -n "$ACCOUNT_KEY" ] && scp_args+=(-i "$ACCOUNT_KEY")
+  run_transport scp "${scp_args[@]}" "$local_path" "$ACCOUNT_TARGET:$remote_path"
+}
+
 dangerous_command_check() {
   local cmd="$1"
   case "$cmd" in
@@ -527,28 +625,65 @@ cmd_doctor() {
 
 cmd_exec() {
   load_project
-  local detach=false cwd="$REMOTE_ROOT" name="" allow_dangerous=false
+  local detach=false cwd="$REMOTE_ROOT" name="" allow_dangerous=false script="" stdin_mode=false remote_script=""
   while [ $# -gt 0 ]; do
     case "$1" in
       --detach) detach=true; shift ;;
       --cwd) cwd="$(remote_join "$2")"; shift 2 ;;
       --name) name="$2"; shift 2 ;;
+      --script) script="$2"; shift 2 ;;
+      --stdin) stdin_mode=true; shift ;;
       --allow-dangerous) allow_dangerous=true; shift ;;
       --) shift; break ;;
       *) break ;;
     esac
   done
-  [ $# -gt 0 ] || die "exec requires a command"
-  local cmd="$*"
+  [ "$stdin_mode" != "true" ] || [ -z "$script" ] || die "Use either --script or --stdin, not both."
+
+  local cmd="" tmp_name result pid status_path remote_cmd detach_rc
+  if [ -n "$script" ]; then
+    local local_script
+    local_script="$(local_join "$script")"
+    tmp_name="$(date +%Y%m%d-%H%M%S)-$(basename "$script" | sed 's/[^A-Za-z0-9_.-]/-/g')"
+    remote_script="$REMOTE_ROOT/.autodl-remote/tmp/$tmp_name"
+    upload_local_file "$local_script" "$remote_script"
+    if [ $# -gt 0 ]; then
+      cmd="$* $(shell_quote "$remote_script")"
+    else
+      cmd="bash $(shell_quote "$remote_script")"
+    fi
+  elif [ "$stdin_mode" = "true" ]; then
+    [ $# -gt 0 ] || die "exec --stdin requires a command after --, for example: exec --stdin -- bash"
+    local tmp_local
+    tmp_local="$(mktemp "${TMPDIR:-/tmp}/autodl-remote-stdin.XXXXXX")" || die "mktemp failed"
+    cat > "$tmp_local"
+    tmp_name="$(date +%Y%m%d-%H%M%S)-stdin.sh"
+    remote_script="$REMOTE_ROOT/.autodl-remote/tmp/$tmp_name"
+    upload_local_file "$tmp_local" "$remote_script"
+    rm -f "$tmp_local"
+    cmd="$* < $(shell_quote "$remote_script")"
+  else
+    [ $# -gt 0 ] || die "exec requires a command"
+    cmd="$*"
+  fi
+
   if [ "$allow_dangerous" != "true" ]; then
     dangerous_command_check "$cmd" || die "Dangerous remote command blocked. Re-run with --allow-dangerous after explicit approval."
   fi
   [ -n "$name" ] || name="exec-$(date +%Y%m%d-%H%M%S)"
-  name="$(printf '%s' "$name" | sed 's/[^A-Za-z0-9_.-]/-/g')"
+  name="$(sanitize_name "$name")"
   local log="$REMOTE_ROOT/.autodl-remote/logs/$name.log"
+  status_path="$REMOTE_ROOT/.autodl-remote/jobs/$name.status"
   info "log=$log"
   if [ "$detach" = "true" ]; then
-    ssh_capture "mkdir -p $(shell_quote "$(dirname "$log")") && cd $(shell_quote "$cwd") && nohup bash -lc $(shell_quote "$cmd") > $(shell_quote "$log") 2>&1 < /dev/null & printf 'pid=%s\nlog=%s\n' \"\$!\" $(shell_quote "$log")"
+    remote_cmd="mkdir -p $(shell_quote "$(dirname "$log")") $(shell_quote "$(dirname "$status_path")") && ( started_at=\$(date +%Y-%m-%dT%H:%M:%S%z); pid=\$BASHPID; printf 'status=running\npid=%s\nstarted_at=%s\nlog=%s\ncwd=%s\n' \"\$pid\" \"\$started_at\" $(shell_quote "$log") $(shell_quote "$cwd") > $(shell_quote "$status_path"); cd $(shell_quote "$cwd") && bash -lc $(shell_quote "$cmd") > $(shell_quote "$log") 2>&1; rc=\$?; finished_at=\$(date +%Y-%m-%dT%H:%M:%S%z); printf 'status=exited\npid=%s\nstarted_at=%s\nfinished_at=%s\nexit_code=%s\nlog=%s\ncwd=%s\n' \"\$pid\" \"\$started_at\" \"\$finished_at\" \"\$rc\" $(shell_quote "$log") $(shell_quote "$cwd") > $(shell_quote "$status_path"); ) < /dev/null & printf 'pid=%s\nlog=%s\nstatus=%s\n' \"\$!\" $(shell_quote "$log") $(shell_quote "$status_path")"
+    result="$(ssh_capture "bash -lc $(shell_quote "$remote_cmd")" 2>&1)"
+    detach_rc=$?
+    printf '%s\n' "$result"
+    [ "$detach_rc" -eq 0 ] || return "$detach_rc"
+    pid="$(printf '%s\n' "$result" | awk -F= '$1=="pid"{print $2; exit}')"
+    [ -n "$pid" ] || die "Remote detach did not return a pid."
+    write_job "$name" "$pid" "$log" "$status_path" "$cwd" "$cmd"
     write_history exec-detach "$cmd log=$log"
   else
     local inner rc
@@ -703,6 +838,159 @@ cmd_tail() {
   ssh_run "tail -n $lines $flag $(shell_quote "$(remote_join "$1")")"
 }
 
+cmd_job_list() {
+  load_project
+  local dir file name pid started log command seen=false
+  dir="$(jobs_dir)"
+  mkdir -p "$dir"
+  if ls "$dir"/*.conf >/dev/null 2>&1; then
+    printf '%-24s %-10s %-24s %s\n' "name" "pid" "started_at" "log"
+    for file in "$dir"/*.conf; do
+      seen=true
+      name="$(job_get "$file" NAME)"
+      pid="$(job_get "$file" PID)"
+      started="$(job_get "$file" STARTED_AT)"
+      log="$(job_get "$file" LOG_PATH)"
+      command="$(job_get "$file" COMMAND)"
+      printf '%-24s %-10s %-24s %s\n' "$name" "$pid" "$started" "$log"
+      [ -n "$command" ] && printf '  command: %s\n' "$command"
+    done
+  fi
+  [ "$seen" = "true" ] || info "No jobs recorded."
+}
+
+cmd_job_status() {
+  load_project
+  [ $# -gt 0 ] || die "job status requires a name"
+  local name file pid log status_path cwd command remote_status rc
+  name="$(sanitize_name "$1")"
+  file="$(job_path "$name")"
+  [ -f "$file" ] || die "Job not found: $name"
+  pid="$(job_get "$file" PID)"
+  log="$(job_get "$file" LOG_PATH)"
+  status_path="$(job_get "$file" STATUS_PATH)"
+  cwd="$(job_get "$file" CWD)"
+  command="$(job_get "$file" COMMAND)"
+  [ -n "$pid" ] || die "Job has no recorded pid: $name"
+
+  info "name=$name"
+  info "pid=$pid"
+  info "cwd=$cwd"
+  info "log=$log"
+  info "status_path=$status_path"
+  info "command=$command"
+  remote_status="$(ssh_capture "if [ -f $(shell_quote "$status_path") ]; then cat $(shell_quote "$status_path"); elif kill -0 $(shell_quote "$pid") 2>/dev/null; then printf 'status=running\npid=%s\n' $(shell_quote "$pid"); else printf 'status=unknown\npid=%s\n' $(shell_quote "$pid"); fi; if kill -0 $(shell_quote "$pid") 2>/dev/null; then echo alive=yes; else echo alive=no; fi" 2>&1)"
+  rc=$?
+  if [ "$rc" -eq 0 ]; then
+    printf '%s\n' "$remote_status"
+  else
+    info "remote_status=unreachable"
+    printf '%s\n' "$remote_status" >&2
+    return "$rc"
+  fi
+}
+
+cmd_job_tail() {
+  load_project
+  local lines=120 follow=false
+  while [ $# -gt 0 ]; do
+    case "$1" in
+      --lines) lines="$2"; shift 2 ;;
+      --follow|-f) follow=true; shift ;;
+      --) shift; break ;;
+      *) break ;;
+    esac
+  done
+  [ $# -gt 0 ] || die "job tail requires a name"
+  local name file log flag=""
+  name="$(sanitize_name "$1")"
+  file="$(job_path "$name")"
+  [ -f "$file" ] || die "Job not found: $name"
+  log="$(job_get "$file" LOG_PATH)"
+  [ "$follow" = "true" ] && flag="-f"
+  ssh_run "tail -n $lines $flag $(shell_quote "$log")"
+}
+
+cmd_job() {
+  [ $# -gt 0 ] || die "job requires a subcommand"
+  local sub="$1"
+  shift
+  case "$sub" in
+    list|ls) cmd_job_list "$@" ;;
+    status) cmd_job_status "$@" ;;
+    tail) cmd_job_tail "$@" ;;
+    *) die "unknown job command: $sub" ;;
+  esac
+}
+
+cmd_shutdown() {
+  load_project
+  local wait_seconds=8 output rc
+  while [ $# -gt 0 ]; do
+    case "$1" in
+      --wait) wait_seconds="$2"; shift 2 ;;
+      *) die "unknown shutdown option: $1" ;;
+    esac
+  done
+
+  info "Requesting remote shutdown for $ACCOUNT_TARGET:$ACCOUNT_PORT"
+  output="$(ssh_capture "sync || true; if command -v shutdown >/dev/null 2>&1; then echo method=shutdown; nohup shutdown >/dev/null 2>&1 </dev/null & elif command -v poweroff >/dev/null 2>&1; then echo method=poweroff; nohup poweroff >/dev/null 2>&1 </dev/null & elif command -v halt >/dev/null 2>&1; then echo method=halt; nohup halt >/dev/null 2>&1 </dev/null & else echo method=missing; exit 127; fi" 2>&1)"
+  rc=$?
+  [ -n "$output" ] && printf '%s\n' "$output"
+
+  if [ "$rc" -ne 0 ] && [ "$rc" -ne 255 ]; then
+    info "shutdown_request=failed rc=$rc"
+    return "$rc"
+  fi
+  if [ "$rc" -eq 255 ]; then
+    info "shutdown_request=connection_closed rc=255"
+  else
+    info "shutdown_request=sent rc=$rc"
+  fi
+
+  sleep "$wait_seconds"
+  if ssh_capture "true" >/dev/null 2>&1; then
+    info "shutdown_status=still_reachable"
+    info "Remote is still reachable; shutdown may not have taken effect."
+    return 1
+  fi
+  info "shutdown_status=disconnected"
+  info "SSH is disconnected; remote is likely shutting down or already powered off."
+  write_history shutdown "wait=$wait_seconds rc=$rc"
+}
+
+cmd_model_dir() {
+  load_project
+  local create=false
+  while [ $# -gt 0 ]; do
+    case "$1" in
+      --mkdir) create=true; shift ;;
+      *) die "unknown model-dir option: $1" ;;
+    esac
+  done
+  if [ "$create" = "true" ]; then
+    ensure_remote_dir "$PROJECT_MODEL_DIR"
+  fi
+  info "$PROJECT_MODEL_DIR"
+}
+
+cmd_put_run() {
+  local uploads=()
+  while [ $# -gt 0 ]; do
+    case "$1" in
+      --) shift; break ;;
+      *) uploads+=("$1"); shift ;;
+    esac
+  done
+  [ "${#uploads[@]}" -gt 0 ] || die "put-run requires at least one local path before --"
+  [ $# -gt 0 ] || die "put-run requires a command after --"
+  local path
+  for path in "${uploads[@]}"; do
+    cmd_put "$path"
+  done
+  cmd_exec -- "$@"
+}
+
 cmd_shell() {
   load_project
   ssh_common_args
@@ -732,13 +1020,17 @@ main() {
     account) cmd_account "$@" ;;
     bind) cmd_bind "$@" ;;
     doctor) cmd_doctor "$@" ;;
+    model-dir) cmd_model_dir "$@" ;;
     exec) cmd_exec "$@" ;;
+    put-run) cmd_put_run "$@" ;;
     put|sync-up) cmd_put "$@" ;;
     get|sync-down) cmd_get "$@" ;;
     tree) cmd_tree "$@" ;;
     ls) cmd_ls "$@" ;;
     cat) cmd_cat "$@" ;;
     tail) cmd_tail "$@" ;;
+    job) cmd_job "$@" ;;
+    shutdown) cmd_shutdown "$@" ;;
     shell) cmd_shell "$@" ;;
     history) cmd_history "$@" ;;
     *) die "unknown command: $cmd" ;;

plugins/autodl-remote/skills/autodl-remote/SKILL.md

@@ -15,6 +15,8 @@ Use this skill when the user wants Codex to work locally while controlling AutoD
 - Codex decides the smallest useful action each time: inspect, upload, download, run, or tail logs.
 - Do not download LLM weights, checkpoints, datasets, or large outputs unless the user explicitly asks.
 - Prefer reading remote files/logs in place with `cat`, `tail`, `ls`, and `tree`.
+- Use detached job commands for long training runs instead of manually guessing process state.
+- Use `shutdown` when the user asks to stop the remote machine; treat SSH disconnect during shutdown as likely success.
 
 ## Setup
 
@@ -38,12 +40,14 @@ The project config should only contain `ACCOUNT` and `REMOTE_ROOT`.
 
 ```bash
 autodl-remote doctor
+autodl-remote model-dir
 autodl-remote tree . --depth 2
 autodl-remote ls .
 autodl-remote cat -- README.md
 autodl-remote tail -- logs/train.log
 
 autodl-remote put ./train.py train.py
+autodl-remote put-run ./train.py -- python train.py
 autodl-remote get outputs/result.csv ./outputs/result.csv
 autodl-remote sync-up ./src src
 autodl-remote sync-down logs/train.log ./logs/train.log
@@ -51,6 +55,12 @@ autodl-remote sync-down logs/train.log ./logs/train.log
 autodl-remote exec -- pwd
 autodl-remote exec -- python train.py
 autodl-remote exec --detach --name train -- python train.py
+autodl-remote exec --script scripts/remote_check.sh
+cat scripts/remote_check.sh | autodl-remote exec --stdin -- bash
+autodl-remote job list
+autodl-remote job status train
+autodl-remote job tail train
+autodl-remote shutdown
 ```
 
 ## Behavior Rules
@@ -60,4 +70,9 @@ autodl-remote exec --detach --name train -- python train.py
 - After remote execution, inspect logs remotely first; pull only small result files when useful.
 - If both local and remote have similar files, do not assume one should overwrite the other. Compare or inspect first, then choose `put` or `get`.
 - Use `exec --detach` for long training jobs and then `tail` the log path.
+- Prefer `job status <name>` and `job tail <name>` for detached jobs created with `--name`.
+- Prefer `exec --script` or `exec --stdin` for complex multi-line commands, Python heredocs, env vars, JSON, awk, or sed.
+- Use `put-run` when the workflow is "upload these local paths, then immediately run this remote command".
+- `model-dir` prints the project model directory convention. The first project command creates `.autodl-remote/CONVENTIONS.md` locally for project notes.
+- `shutdown` first syncs the remote filesystem, then tries provider-safe shutdown methods, waits, and verifies that SSH disconnects.
 - Treat broad destructive commands as dangerous; require explicit user approval before using `--allow-dangerous`.