experiment-designer

# Experiment Designer Turn one risky question into one experiment whose verdict you'll actually believe — because you wrote down what would change your mind *before* you looked at the result. The one rule everything serves: > **Fix the pass/fail bar before the run. Then don't move it.** That single discipline is the difference between a test that informs a decision and a number you rationalize past. The most creditable thing a doomed project can do is die on a bar it set in advance; the most common way good evidence gets wasted is moving the bar once the number is in. --- ## Why this exists When the result arrives, you are the most biased you will ever be about it — you've now got effort, hope, and a half-built thing riding on it. A threshold chosen *after* seeing data isn't a threshold, it's a justification. So this skill front-loads every judgment call (what we're testing, what counts as pass, what would kill it) into a moment when you have no result yet and therefore nothing to defend. Two failure modes it specifically targets: - **The un-falsifiable hypothesis.** "I think this will help" with no number and no fail branch can't be run, only believed. If you can't say what result would make you *abandon* the idea, you don't have an experiment. - **The confounded test that lies.** An optimization tuned for speed can ship a confident wrong answer if correctness was never the controlled baseline. A test that can pass while being broken is worse than no test. ## When t...