evaluation-methodology

# Evaluation Methodology This document is the authoritative reference for how PluginEval measures plugin and skill quality. It covers the three evaluation layers, all ten scoring dimensions, the composite formula, badge thresholds, anti-pattern flags, Elo ranking, and actionable improvement tips. Related: [Full rubric anchors](references/rubrics.md) --- ## The Three Evaluation Layers PluginEval stacks three complementary layers. Each layer produces a score between 0.0 and 1.0 for each applicable dimension, and later layers override or blend with earlier ones according to per-dimension blend weights. ### Layer 1 — Static Analysis **Speed:** < 2 seconds. No LLM calls. Deterministic. The static analyzer (`layers/static.py`) runs six sub-checks directly against the parsed SKILL.md: | Sub-check | What it measures | |---|---| | `frontmatter_quality` | Name presence, description length, trigger-phrase quality | | `orchestration_wiring` | Output/input documentation, code block count, orchestrator anti-pattern | | `progressive_disclosure` | Line count vs. sweet-spot (200–600 lines), references/ and assets/ bonuses | | `structural_completeness` | Heading density, code blocks, examples section, troubleshooting section | | `token_efficiency` | MUST/NEVER/ALWAYS density, duplicate-line repetition ratio | | `ecosystem_coherence` | Cross-references to other skills/agents, "related"/"see also" mentions | These six sub-checks feed directly into six of the ten final dimensions (via `...