count-exact-rows-before-you-fit-a-correction

# Count the rows you get exactly right, then decide what to do about the rest Two runs report a mean absolute error near 1e-3 on the same 5,000-row split. - **Run A** reproduces 4,987 rows to better than 1e-6 and misses 13, by about 0.1 to 0.4 each. Its remaining work is thirteen rows long and each one has a nameable cause. - **Run B** reproduces none of them, is off by ~1e-4 in the middle of the distribution and ~1e-2 in the tail, and its remaining work is a five-thousand-row approximation problem. Those are opposite situations demanding opposite next actions, and the metric being scored cannot tell them apart — A scored 0.00069, B scored 0.00153, a factor of two on a task where the gap in *understanding* is total. Both are real: they are two arms on one task, and B lost. One number separates them, and it is not the mean. ## Make the exact fraction the progress metric **Set the tolerance from the residuals, not from taste.** Sweep it by decades and look for a plateau — a stretch where the count does not move is the gap between "right" and "wrong", and anywhere inside it is the right tolerance: ```python a = np.abs(resid) print(f"MAE {a.mean():.3e} median {np.median(a):.3e} max {a.max():.3e}") for e in range(-9, 1): # count under each decade print(f" |r| < 1e{e}: {(a < 10.0**e).sum()}") for e in range(-9, 1): # and how many live in each decade print(f" 1e{e}..1e{e+1}: {np.sum((a >= 10.0**e) & (a < 10.0**(e+...