skill-eval-setuplisted

# Skill Eval Setup Takes a skill name or path, reads its SKILL.md, and produces: 1. An `evals/` folder inside the skill directory 2. `evals/evals.json` — true/false assertions auto-derived from the SKILL.md 3. `evals/test_scenarios.json` — 2–3 synthetic scenarios to test the skill against 4. A copy-paste prompt the user runs in a fresh window to start the self-improvement loop --- ## Input The user provides one of: - A skill name (e.g., `session-wrap`) — resolved to `.claude/skills/<name>/SKILL.md` - A relative or absolute path to a skill folder or SKILL.md Optional flag: - `--keep-scenarios` — when the target skill already has `evals/test_scenarios.json`, do NOT overwrite it. Regenerate only `evals.json`. Use this when regenerating the assertion set against a rebuilt eval bar while preserving hand-crafted, project-specific scenarios. If the skill or SKILL.md cannot be found, report the error and stop. --- ## Step 1: Read and analyze the SKILL.md Read the target SKILL.md completely. Extract every testable requirement: ### What to look for | Signal | Example | Assertion type | |---|---|---| | **Required sections** | "Output two parts: X and Y" | Structure — output contains X and Y | | **Format rules** | "wrapped in a code fence labeled `text`" | Structure — format matches | | **Quantity constraints** | "4–8 lines", "2–5 files", "300–600 words" | Structure — count in range | | **Content requirements** | "must be self-contained", "include the magic word" | Content — s