← ClaudeAtlas

evaluating-skillslisted

Use when defining eval coverage for a skill, authoring or maintaining its eval suite, or assessing whether a skill or revision improves agent behavior.
slowdini/slow-powers · ★ 2 · AI & Automation · score 68
Install: claude install-skill slowdini/slow-powers
# Evaluating skills A skill exists to solve a problem or improve a process. Its eval suite measures whether it achieves those goals, and keeps measuring them as the prose changes. Coverage belongs to the goals; a passing score needs evidence of useful work. `slow-powers:writing-skills` owns drafting the skill. This skill owns evaluation design and interpretation. [eval-magic](https://github.com/slowdini/eval-magic) owns environments, dispatch, grading, and artifacts. Use its help and generated runbook for execution. ## Evaluation checklist Copy the items needed for the agreed scope into your persistent task tracker. A suite draft, an exploratory run, and a validated baseline are different deliverables; name which one you are producing and leave unfinished measurement explicit. - [ ] State the motivating problem, intended goals, and observable success. - [ ] Reuse the suite or identify gaps in goal coverage before authoring cases. - [ ] Choose a realistic environment, comparison, model/harness population, and run budget. - [ ] Screen likely ceilings and check subject isolation before dispatch. - [ ] For new cases, inspect exploratory evidence before settling the assertions. - [ ] Check that graders accept valid solutions and reject relevant failures. - [ ] Measure the frozen suite across fresh sessions and inspect validity, outcomes, and cost. - [ ] Record results, limitations, and coverage gaps alongside the suite. ## Start with the skill's goals Before drafting a skil