← ClaudeAtlas

experimental-designlisted

Best practices for designing reproducible ML experiments. Use when planning ablations, baselines, or controlled experiments.
thada2402/AutoResearchClaw · ★ 1 · AI & Automation · score 75
Install: claude install-skill thada2402/AutoResearchClaw
## Experimental Design Best Practice 1. ALWAYS include meaningful baselines (not just random): - At least one classical method baseline - At least one recent SOTA method baseline - A simple-but-strong baseline (e.g., linear probe, k-NN) 2. Use MULTIPLE random seeds (minimum 3, ideally 5) 3. Report mean +/- std across seeds 4. Design ablations that isolate EACH key component: - Remove one component at a time - Each ablation must be meaningfully different from baseline 5. Control variables: change only ONE thing per comparison 6. Use standard splits (train/val/test) — never test on training data 7. Report wall-clock time and memory usage alongside accuracy