Overview
Mental model for the Yale SOM HPC cluster — login vs compute nodes, partitions, GPFS, Slurm, default safety rules. TRIGGER when the user is working on the Yale SOM HPC cluster and needs orientation, is deciding where work belongs on the cluster, or needs high-level HPC workflow guidance before reaching for a task-specific skill.
Using The Filesystem
Use GPFS on the Yale SOM HPC cluster (/gpfs/project, /gpfs/scratch60, compute-node /tmp) without metadata storms. TRIGGER when choosing storage locations on the Yale SOM HPC cluster, moving files to/from GPFS, using cluster scratch/tmp, handling many small files on the cluster, or diagnosing GPFS I/O bottlenecks.
Scraping At Scale
Build resumable, durable caches for high-volume web scraping/crawling on the Yale SOM HPC cluster — SQLite WAL catalogs, batched writers, single-archive body storage, and /local staging — without a GPFS metadata storm. TRIGGER when scraping or crawling many thousands of pages, building a resumable fetch catalog/cache, or storing large numbers of web artifacts on the cluster.
Code Review
Review code, diffs, or PRs for correctness, plus research/cluster pitfalls — hardcoded scratch paths, resource mismatch, GPFS tiny-file storms, package installs inside arrays. TRIGGER when reviewing code or a diff/PR, or before committing.
Programming And Coding
Cross-cutting research-coding principles. TRIGGER when writing or reviewing research code.
Coding In R
How to write R well — renv, tidyverse/data.table, project paths, style, scripts, seeds. TRIGGER when authoring or editing .R/.Rmd/.qmd files. For running R on the Yale SOM HPC cluster (Slurm, renv on /gpfs), use running-r instead.
Acquiring Data
Download, query, scrape, and call APIs from the Yale SOM HPC cluster without leaking credentials, repeating expensive requests, or getting the shared outbound IP blocked. TRIGGER when fetching datasets onto /gpfs, calling WRDS/REST APIs, scraping, caching downloads, or handling credentials on the cluster.
Using Gpus
Request GPUs on the Yale SOM HPC cluster only when code actively uses them, and diagnose idle allocations. TRIGGER when writing GPU sbatch scripts for the Yale SOM HPC cluster, running CUDA/PyTorch/JAX/TensorFlow/RAPIDS jobs there, or checking nvidia-smi inside a cluster GPU allocation.
Task Runner
Capture a project's commands as named, rerunnable recipes — via shell script, Makefile, or justfile — so analysis steps are reproducible and the agent can rerun them. TRIGGER when automating or documenting project commands, writing a justfile/Makefile/run script, or choosing among shell vs make vs just.
Using Git And Github
Git and GitHub for research work — commits, branches, what to track, big-file pushback, PRs; on the Yale SOM HPC cluster also module load git and SSH agent forwarding. TRIGGER when running git/gh, committing, deciding what to track or ignore, or opening PRs.
Installing Software
Install software on the Yale SOM HPC cluster with Lmod modules, uv, static/musl binaries, and Apptainer — no sudo. TRIGGER when installing tools or packages on the Yale SOM HPC cluster, hitting GLIBC errors on cluster nodes, building from source on the cluster, or using Apptainer containers there.
Running R
Run R on the Yale SOM HPC cluster with Lmod modules, renv, batch scripts, and BLAS/OpenMP thread control. TRIGGER when writing R Slurm jobs on the Yale SOM HPC cluster, using renv on the cluster, installing R packages on the cluster, or running Rscript in batch mode there.
Staying Connected
Keep work alive across dropped SSH connections to the Yale SOM HPC cluster, and decide where Claude Code itself runs (login node vs. your laptop driving the cluster). TRIGGER when a cluster session dies on sleep/wifi drop, running Claude Code or a long agent against the cluster, keeping tmux/zmx sessions alive there, or a long-running shell losing its work on disconnect.
Managing Jobs
Submit, monitor, cancel, array, and chain Slurm jobs on the Yale SOM HPC cluster. TRIGGER when running, submitting, or scheduling any job or analysis on the cluster (even phrased plainly, e.g. "run my regression on the cluster"), writing sbatch scripts, choosing partitions/resources, or using job arrays, dependencies, or sacct/squeue/scancel.
Self Diagnosing Resource Use
Diagnose whether a Yale SOM HPC cluster Slurm job used its requested CPUs, memory, GPUs, and time wisely, then right-size the next request. TRIGGER when a Slurm job on the Yale SOM HPC cluster is slow, killed, pending, idle, over-requested, ran out of memory, or failed, or whenever a job just finished and should be checked for waste.
Running Python
Run Python on the Yale SOM HPC cluster with uv, Slurm, thread control, logging, and resumable outputs. TRIGGER when writing Python sbatch scripts for the Yale SOM HPC cluster, creating uv environments under /gpfs, or debugging Python Slurm jobs on the cluster.
Starting A New Project
Create a research project layout under /gpfs/project on the Yale SOM HPC cluster — reproducible, resumable, safe for shared use. TRIGGER when starting or reorganizing a project on the Yale SOM HPC cluster, choosing GPFS directories, or setting up cluster-side logs, lockfiles, and Slurm scripts.
Running Stata
Run Stata batch jobs on the Yale SOM HPC cluster with logs, scratch temp, CPU matching, and license courtesy. TRIGGER when writing Stata sbatch scripts for the Yale SOM HPC cluster, choosing Stata/MP cores there, or diagnosing Stata batch jobs/licenses on the cluster.
Parallel Python
Choose threads, processes, queues, and Slurm arrays for Python on the Yale SOM HPC cluster. TRIGGER when running multiprocessing/concurrent.futures/joblib/Dask/Ray on the cluster, writing Slurm array jobs there, or sizing workers to SLURM_CPUS_PER_TASK.
Connecting Securely
Connect to the Yale SOM HPC cluster (hpc.som.yale.edu) over SSH with keys, agents, jump hosts, and tunnels — no copied private keys. TRIGGER when SSHing to the Yale SOM HPC cluster, setting up VS Code/Jupyter tunnels to a cluster compute node, forwarding ports through the cluster login node, or troubleshooting cluster SSH/agent auth.
Coding In Python
How to write Python well — uv environments, formatting, CLI scripts, logging, seeds. TRIGGER when authoring or editing .py files. For running Python on the Yale SOM HPC cluster (sbatch, Slurm, uv on /gpfs), use running-python instead.
Troubleshooting
Triage Yale SOM HPC cluster errors and map each to a fix. TRIGGER when a cluster job fails, is killed, or pends unexpectedly, or the user pastes a cluster error (GLIBC not found, no package called X, ModuleNotFoundError, command not found, bad interpreter, $'\r', nvidia-smi failure, empty Stata .out).
Accelerating Python
Make slow or memory-bound Python fast on the Yale SOM HPC cluster — profile, push work into DuckDB/Polars/Arrow, vectorize/Numba, then parallelize or use a GPU. Also covers data larger than a job's RAM (Parquet, query engines, sampling). TRIGGER when a Python job on the cluster is slow, CPU-bound, or memory-heavy, when a dataset on /gpfs does not fit a job's RAM, or when weighing parallelism or G…