rseng-hpc-computing

# Research computing on HPC clusters HPC clusters are where much research code actually runs, and they invert desktop habits: you do not run programs, you request resources and submit batch jobs; you do not install software, you load modules or bring containers; the login node is a shared hallway, not a workstation. An agent's job is to translate the user's computation into this model correctly and reproducibly - and to respect that every cluster has local documentation that overrides generic advice. ## Batch jobs done right (SLURM) A job script is a shell script with resource directives: ```bash #!/bin/bash #SBATCH --job-name=fit-model #SBATCH --time=02:00:00 #SBATCH --nodes=1 #SBATCH --ntasks=1 #SBATCH --cpus-per-task=8 #SBATCH --mem=16G #SBATCH --output=logs/%x-%j.out module load Python/3.12 srun python fit_model.py --config configs/run.yaml ``` Practices that matter: - Request honestly: measure a pilot run, then set time/memory with modest headroom. Over-requesting wastes allocation and queue priority (and energy - rseng-green-computing); under-requesting kills jobs at 99%. - Never compute on the login node; test with a short interactive allocation (srun --pty or salloc) instead. - Parameter sweeps are job arrays (--array=0-99), not 100 submitted scripts; index into a config list with $SLURM_ARRAY_TASK_ID (externalized run configs pair naturally with rseng-workflows). - Long jobs checkpoint: clusters preempt and nodes fail; save restartable state at i...