skill-eval
FeaturedMeasure whether a skill helps a named task or needs revision or removal. Use when: a bounded routing or coding evaluation is requested; conformance alone cannot show benefit.
Install
Quality Score: 91/100
Skill Content
Details
- Author
- boshu2
- Repository
- boshu2/agentops
- Created
- 10 months ago
- Last Updated
- today
- Language
- Go
- License
- Apache-2.0
Bundled in these plugins
Similar Skills
Semantically similar based on skill content — not just same category
skill-evaluator
Design and evaluate Agent Skill experiments that measure whether a skill improves model behavior, discovery, task success, reliability, cost, or latency relative to a no-skill/prior-version baseline, including host/model comparisons, judge agreement, quality-cost Pareto trade-offs, and runtime drift under frozen measurement identity. Use when the user asks to benchmark, A/B test, evaluate, compare, prove, regress-test, or measure a skill across supported harnesses, or to prepare executable real-host suites when execution is unavailable. Do not use for static package/routing audits (use skill-auditor), to create/edit a skill (use skill-creator), to fabricate real-host results, or to claim universal superiority from one configuration.
skill-eval
Use when you want to test, measure, or compare skill effectiveness. Supports eval, compare, metrics, baseline capture, and regression detection.
skill-eval-harness
Write, evaluate and maintain agent skills with measured evidence rather than assertion. Covers SKILL.md structure and frontmatter, writing a description that triggers reliably without firing on neighbouring topics, progressive disclosure through references and scripts, and building eval suites across three families - usefulness with skill on versus off, discovery, and adversarial discovery. Use when creating a new skill, reviewing or debugging one that does not trigger or does not help, deciding whether a skill earns its context cost, writing an eval suite or grader, or interpreting skill eval results. Also use when asked whether a skill is any good.