rseng-numerical-accuracylisted
Install: claude install-skill fdiblen/rseng-agent-skills
# Numerical accuracy in research code
Floating-point numbers are approximations with well-defined rules,
and research conclusions can hinge on respecting them. The two
recurring failures: treating floats as exact (== comparisons,
accumulating error blindly) and treating all differences as noise
(tolerances loosened until tests pass). Both are avoidable with a
small set of habits - and both matter more in research than
elsewhere, because the numbers ARE the result.
## The ground rules
- Never compare floats with == (except against an exactly
representable sentinel like 0.0 you assigned yourself). Use
tolerance-based comparison: `math.isclose`, `numpy.isclose`, or
the testing helpers below.
- Decimal literals are usually not representable: 0.1 is stored as
the nearest binary fraction, which is why 0.1 + 0.2 != 0.3.
Format-rounding for display hides this; arithmetic does not.
- NaN propagates and never equals anything, including itself; test
with isnan, and decide explicitly whether NaN in data means
missing, invalid or bug (rseng-data-management's missing-data
discipline). Silent NaN propagation into published numbers is the
classic silent failure.
- Precision-degrading operations (subtracting nearly equal numbers -
catastrophic cancellation; summing numbers of very different
magnitude) lose precision structurally; restructure the formula
(e.g. use expm1/log1p, two-pass variance algorithms) rather than
adding digits.
## Tolerances: choose, do not t