eval-iteratelisted
Install: claude install-skill mthines/agent-skills
# Eval Iterate
Drive a failing AI/LLM eval to a real green: diagnose, fix, re-run,
confirm — capped at 5 iterations, never by weakening the eval.
This `SKILL.md` is the **orchestration index**.
Load the matching rule file when you need detail — do not preload them.
| Phase | Goal | Required rule |
| ----- | ---- | ------------- |
| 0 | Resolve the target eval + capture the baseline failure | this file |
| 1 | Resolve how to run it | this file |
| 2 | Classify the failure (verdict required) | [`rules/eval-bug-classification.md`](./rules/eval-bug-classification.md) |
| 3 | Apply the minimal fix — gated if it touches the eval itself | [`rules/anti-gaming-guard.md`](./rules/anti-gaming-guard.md) |
| 4 | Re-run, then confirm with a second run | [`rules/convergence-confirmation.md`](./rules/convergence-confirmation.md) |
| 5 | Iterate or stop at the cap | this file |
| 6 | Report (structured exit summary) | this file |
Always read [`rules/anti-gaming-guard.md`](./rules/anti-gaming-guard.md)
before touching any eval definition (assertion, threshold, golden-set
item, judge prompt). The refusals in it apply on every iteration.
## Input
The user provides one of:
- An eval identifier — an L2 suite name (e.g. `tier-routing`), a golden-set
file path, or a test/eval file path.
- A PR URL with a failing eval check.
- **Nothing** — if `$ARGUMENTS` is empty, auto-detect the failing eval
check on the current branch's open PR (see Phase 0).
- `--max-iterations <n>` — lowers the cap b