reproducibility-first

# Reproducibility-First ## Overview **One observation is not a gradient.** An LLM call is one sample of a distribution. A CI run is one sample of an intermittent fault. A failing test on your machine is one sample of your environment. Acting on a single sample is not SGD — it's noise injection. **Core principle:** before you use a failure as the basis for a code edit, you must either **(a)** reproduce it at least once more, or **(b)** prove the log is unambiguous signal that does not require repetition. See [`../../docs/ldd/convergence.md`](../../docs/ldd/convergence.md) §3.3 "Noisy SGD" for why this matters for convergence. ## When to Use Invoke this skill **before** `root-cause-by-layer` whenever: - A test failed once in CI but passed locally (or vice versa) - An LLM returned a malformed response once - A log message looks surprising but you can't explain why - A gate rejected on something that seemed fine in review - An "it's flaky" narrative is forming around a specific case - You're about to retry "and see what happens" ## The Two Branches ### Branch A — Reproduce before editing Run the failing case at least **twice more** in an environment as close as possible to the one that failed. Possible outcomes: | N additional runs | Pattern | Interpretation | Action | |---|---|---|---| | 2 of 2 pass | transient | noise — **do not edit code** | Log the incident, check infra / rate limits / upstream outage, close | | 1 of 2 fails | flaky, rate ≥ 33% | real but intermitt...