quant-validation

# Validating a financial model — the five ways the number lies A backtest that looks excellent and loses money live is not usually a bad strategy. It is a good measurement of the wrong thing. Each section below is one mechanism by which a number becomes convincing without becoming true. **On sourcing.** The methods here are standard and attributable — most of them to Marcos López de Prado's *Advances in Financial Machine Learning*, with the information-ratio framing from Grinold & Kahn. This file states the MECHANISM and what to check, and deliberately does not restate formulas from memory. Where an implementation needs an exact expression — the deflated Sharpe ratio in particular — verify it against the primary source before shipping a number that depends on it. A formula recalled approximately is worse here than no formula: it produces a specific, wrong, confident figure. ## 1. Purged cross-validation with an embargo **The leak.** In a normal k-fold split, training and test rows are disjoint. In a financial series they are not independent: a label at time *t* is computed from data spanning *t* to *t+h*. A training observation inside that window has seen the future the test observation is being asked to predict. **Purging.** Drop from the training set every observation whose label window overlaps the label window of any test observation. Not the observation's timestamp — its *label window*. This is the step people skip, because a plain timestamp split looks like it alre...