skill-benchmark-gate

Solid

Use when a skill change is about to ship and must pass a release gate. Validates gate inputs fail-closed, runs activation and regression cases against an explicit baseline, measures context cost against an explicit ceiling, and emits PASS or FAIL with one issue per failure. Not for scoring without gating — use skill-benchmark.

AI & Automation 35 stars 0 forks Updated 2 days ago Apache-2.0

Install

View on GitHub

Quality Score: 82/100

Stars 20%
52
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# Skill benchmark gate ## Contract | Field | Bound contract | |---|---| | Trigger | Before shipping any skill change; diagnosing weak skill activation, regressions, or context cost; setting release gates for skill packs | | Authority | Read and evaluate the skill; write only gate logs and issue artifacts; do not ship the change | | Side effect | Runs evaluation loops; writes run-log entries; opens follow-up issues on GitHub or equivalent tracker | | Done | A reproducible PASS or FAIL record with case results, deltas, context cost, and failure issues | ## Inputs All inputs are required and must be explicit. No defaults are inferred from the procedure. 1. Changed skill and comparison revision: the skill slug, the diff or branch under test, and the baseline revision to compare against. 2. Explicit universal activation case set: the complete list of cases that must all show activation greater than 0% with the skill enabled. Every case must be named; no case is implied. 3. Explicit critical regression case set and baseline: the list of critical scenarios and their recorded baseline results (pass/fail and context token count per scenario) from the comparison revision. Every scenario must be named with its baseline; no scenario is implied. 4. Explicit context-budget ceiling: the maximum total context tokens the skill may load, stated as a number. 5. Explicit iteration budget: the maximum number of evaluation iterations the gate may run, stated as a number. ## Procedure 1. Val...

Details

Author
OutlineDriven
Repository
OutlineDriven/odin-claude-plugin
Created
9 months ago
Last Updated
2 days ago
Language
Python
License
Apache-2.0

Similar Skills

Semantically similar based on skill content — not just same category