flaky-detectlisted
Install: claude install-skill QAIA-Project/QAIA
# flaky-detect — flaky test detection from repeated runs
A test whose verdict flips between runs of unchanged code destroys the signal value of the whole
suite: every red becomes negotiable, and a real regression hides behind "just re-run it". This
skill names those tests with evidence, and stops there — it never retries, quarantines or fixes.
The defect class is not hypothetical: QAIA's own automation has hit it (shared mutable state
raced by parallel workers). For the provenance and the audited evidence trail, see
`references/origin.md` — not needed to run the skill.
Reference fixture: `fixture/` in this skill folder — a minimal, self-contained Playwright
suite against a shared-state server, deliberately timing-dependent (not "always fails"),
whose 5 captured real runs (`fixture/runs/results-run{1..5}.xml`) show genuine pass/fail
variance per test with **zero code changes between runs**. See `fixture/VALIDATION.md` for
the worked example of this skill applied to that data.
## Input
- **N ≥ 3 runs** (recommended) of the **same test code**, same suite, in the formats
`run-report` already produces: JUnit XML and/or Cucumber JSON. 2 runs is the visible floor —
enough to see *a* difference, never enough to trust it isn't a one-off. **1 run can never
show flakiness at all**: a single green run only proves "passed this once," never "is
stable" — say so rather than implying otherwise.
- The runs must span **no code change to the test or the SUT** between them — that is