monitor-experiment

# Monitor Experiment Results > ⏱ **External cadence is appropriate here.** This skill waits on an external > fact (job completion / progress), so it is a natural `/loop` / `CronCreate` > surface: the wake reads status and self-judges only **machine-checkable** > completion (exit code, file exists, epoch logged) — never quality. This is > the additive external-wait shape in > [`shared-references/external-cadence.md`](../shared-references/external-cadence.md). > If a scheduled wait here ends in a verdict step (e.g. then audit results), > run that verdict **once** after the wait clears — not re-entered per tick. Monitor: $ARGUMENTS ## Workflow ### Step 1: Check What's Running **SSH server:** ```bash ssh <server> "screen -ls" ``` **Vast.ai instance** (read `ssh_host`, `ssh_port` from `vast-instances.json`): ```bash ssh -p <PORT> root@<HOST> "screen -ls" ``` Also check vast.ai instance status: ```bash vastai show instances ``` **Modal** (when `gpu: modal` in CLAUDE.md): ```bash modal app list # List running/recent apps modal app logs <app> # Stream logs from a running app ``` Modal apps auto-terminate when done — if it's not in the list, it already finished. Check results via `modal volume ls <volume>` or local output. ### Step 2: Collect Output from Each Screen For each screen session, capture the last N lines: ```bash ssh <server> "screen -S <name> -X hardcopy /tmp/screen_<name>.txt && tail -50 /tmp/screen_<name>.txt" ``` If hardcopy fails, check for log file...