benchmark-design
FeaturedDesign and calibrate a multi-Case capability Benchmark and establish a traceable Formal Baseline.
Install
Quality Score: 93/100
Skill Content
Details
- Author
- Prism-Shadow
- Repository
- Prism-Shadow/penguin-harness
- Created
- 2 months ago
- Last Updated
- today
- Language
- TypeScript
- License
- Apache-2.0
Similar Skills
Semantically similar based on skill content — not just same category
benchmark-curator
Design, curate, version, and quality-control benchmark or evaluation corpora for Agent Skills and quality workflows, including task taxonomies, discovery/forced/negative controls, adversarial and regression cases, difficulty strata, holdout isolation, provenance, duplication, contamination risk, coverage balance, and immutable benchmark hashes. Use when the user asks to build or maintain an eval dataset, golden set, benchmark suite, regression corpus, holdout, challenge set, or representative test cases. Do not use to execute the model experiment or claim lift (use skill-evaluator), to design the grading rubric (use rubric-designer), to edit the candidate skill (use skill-creator), or to treat leaked/known cases as a clean holdout.
agent-evaluation
Run one specified Test Agent on one specified Benchmark Case exactly once, privately score that execution, and return one protocol result.
benchmark-implement
Run an approved PRD through the existing receipt-backed implement workflow, analyze its runtime session with a fresh independent evaluator, emit a deterministic process report, and compare it with an earlier run. Use when the user invokes "$benchmark-implement", asks to benchmark the implement skill or harness, rerun a fixed PRD after harness changes, analyze implementation-session efficiency, or compare baseline and candidate harness behavior.