evaluate-skilllisted

Use when you want to behaviorally evaluate a Claude Code skill — measure whether it auto-activates on the prompts it should and stays quiet on near-misses (trigger recall and specificity), whether its output actually satisfies its own discipline (correct-usage), and whether it beats the no-skill baseline (with/without) — producing a scorecard with confidence intervals. Triggers on "evaluate this skill", "test my skill", "does my skill fire", "measure or benchmark skill performance", "is my description triggering", "build an eval for this skill", or running "/evaluate-skill". Claude Code only — it spawns many headless `claude -p` runs, so it is cost-gated; show the plan first. Not for a one-off manual spot-check, for judging a skill's design qualitatively (that is a fresh-eyes panel, not a behavioral eval), or for listing which skills are installed (that is toolkit-awareness).
grimaldost/craft-collection · ★ 2 · AI & Automation · score 71

Install: claude install-skill grimaldost/craft-collection

# Evaluate Skill Behaviorally evaluate a Claude Code skill — not by reading it (that is a design review), but by **running it headless many times and measuring what it does**. The engine that produced this collection's own scorecards ships in `scripts/`; this skill points it at any skill. ## What it measures — three axes 1. **Triggering** — does the skill auto-activate on prompts it should (recall) and stay quiet on near-misses it shouldn't (specificity)? Each prompt is run headless with the plugin loaded; the run is a hit if the Skill tool fired. 2. **Correct-usage** — when it fires, does the output satisfy the skill's own discipline? An LLM judge scores the WITH-skill output against a rubric you write. 3. **With/without** — does the skill beat the no-skill baseline? A swap-order pairwise judge compares WITH vs WITHOUT output — a win counts only if *both* orderings agree, else it is a tie (this cancels position bias). Every rate carries a **Wilson 95% CI**, so you can tell signal from small-N noise. ## When to use - You wrote or edited a skill and want to know if it actually triggers and helps. - A description changed and you need to catch a triggering regression. - You're choosing between two phrasings of a description. **Not for:** a one-off manual check (just try it), or judging a skill's *design* qualitatively — bloat, correctness, redundancy — which is a fresh-eyes review (see the `review-panel` skill), not a behavioral eval. The two answer differe