← ClaudeAtlas

flaky-test-investigationlisted

Use when tests pass and fail intermittently without code changes, or a test passes alone but fails in the full suite.
yeaight7/agent-powerups · ★ 7 · Testing & QA · score 75
Install: claude install-skill yeaight7/agent-powerups
## Purpose Flaky tests erode trust in CI. Do not just re-run them and hope for the best — isolate the flake vector, fix it, and prove the fix with a stress loop. ## When to Use - A test fails intermittently in CI but passes locally (or vice versa) - A test passes alone but fails in the full suite - A re-run "fixed" a failure and nobody knows why ## Inputs - The flaky test's name/path and the runner command for it - Recent failing runs, if available, to estimate the failure rate ## Workflow 1. **Isolate the test.** Run the specific failing test by itself. If it passes alone, the flake is likely an **order dependency** or **state leakage** from a previous test — run the suite up to and including it to confirm. 2. **Stress test.** Run the test in a tight loop to establish the failure rate before changing anything: ```bash for i in {1..100}; do npm test -- -t "My Test" || echo "FAIL on run $i"; done ``` (Adapt the inner command to the project's runner; some runners have repeat flags built in.) 3. **Check the common vectors:** - **Time** — does the test rely on `Date.now()` or `setTimeout`? Mock the clock. - **Async/Promises** — asserting before a background task finishes? Ensure proper `await` or `waitFor` usage. - **Shared state** — reusing database records, global singletons, or mutated variables between runs? Ensure clean teardowns in `afterEach`. - **Randomness** — random IDs or sort orders? Force deterministic seeds or sort orders. 4. **Pr