skill-evaluation

Featured

Evaluate any agent skill against a merged framework — Anthropic's Claude Code best practices plus Matt Pocock's writing-great-skills methodology — across 4 axes (Trigger, Structure, Steering, Pruning). Produces an evidence-cited scorecard (0–100), a weighted overall score, and diagnosed failure modes with prioritized fixes. Use when the user asks to evaluate, rate, or audit a skill ("evaluate this skill", "skill scorecard", "review SKILL.md"), or to compare two skills.

AI & Automation 96 stars 8 forks Updated 4 days ago Apache-2.0

Install

View on GitHub

Quality Score: 88/100

Stars 20%
66
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
80
License 10%
100
Description 5%
100

Skill Content

# Skill Evaluation If you need the vocabulary and tests behind Axes 1, 3, and 4 (leading words, completion criteria, context pointers, the deletion test, failure-mode definitions), read `references/mechanics.md` before scoring those axes. ## Source - [Lessons from building Claude Code: How we use skills](https://claude.com/blog/lessons-from-building-claude-code-how-we-use-skills) — Anthropic, Jun 2026 - "The Missing Manual: How to Write Great Skills" — Matt Pocock, AI Engineer World's Fair 2026 ([video](https://www.youtube.com/watch?v=UNzCG3lw6O0)), and his `writing-great-skills` skill ## Parameters | Parameter | Description | Default | |-----------|-------------|---------| | `target` | Path to skill directory or SKILL.md to evaluate | Ask user | | `output` | Path to write the scorecard | `<target>/EVALUATION.md` | | `compare` | Optional second skill to compare side-by-side | None | Also runs unattended: in CI, point `target` at skills changed in a PR and gate with `scripts/score.py --fail-below 60 ...` — non-zero exit below the threshold fails the check. ## Criteria 18 criteria: 14 core, scored on every skill, plus 4 conditional criteria scored only when the skill's category makes them apply — otherwise mark **N/A** and exclude the criterion from both the numerator and denominator of the weighted average. Every score is 0–100 with evidence citing file, section, or line. ### Axis 1 — Trigger (invocation) | # | Criterion | Weight | Key question | |---|-----------|---...

Details

Author
fabricioctelles
Repository
fabricioctelles/skills
Created
7 months ago
Last Updated
4 days ago
Language
Python
License
Apache-2.0

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category

AI & Automation Listed

skill-evaluation-graph

Deeply analyze, audit, score, and optimize agent skills conforming to the Agent Skills standard or Antigravity/Codex/Claude formats. Use when reviewing an existing skill, diagnosing why an agent misfires or burns context, pruning skill bloat, refactoring monolithic instructions into progressive disclosure, or benchmarking skill quality. Do not use for creating skills from scratch without an existing procedure (use workflow-skill-creator).

2 Updated 2 weeks ago
MaxLaurieHutchinson
Code & Development Listed

skill-eval-harness

Write, evaluate and maintain agent skills with measured evidence rather than assertion. Covers SKILL.md structure and frontmatter, writing a description that triggers reliably without firing on neighbouring topics, progressive disclosure through references and scripts, and building eval suites across three families - usefulness with skill on versus off, discovery, and adversarial discovery. Use when creating a new skill, reviewing or debugging one that does not trigger or does not help, deciding whether a skill earns its context cost, writing an eval suite or grader, or interpreting skill eval results. Also use when asked whether a skill is any good.

0 Updated 4 days ago
Premchand006
AI & Automation Listed

skill-eval-loop

Measure whether an agent skill actually helps: prompt sets with should/should-not trigger, with/without skill runs, deterministic checks, rubric grading, CI regressions. Use when hardening a SKILL.md, deciding if a rewrite improved quality, or adding a quality gate for skills.

1 Updated today
novahiz