benchmarklisted

# Benchmark — epic-harness A/B Evaluation & Golden Set Runner **CRITICAL**: This is a repository-local developer evaluation skill. It is **NOT** included in the published `epic-harness` plugin bundle to prevent runtime context overhead in user projects. --- ## 1. When to Trigger - Explicit `/benchmark`, `/benchmark full`, `/benchmark smoke`, or `/benchmark guard` commands - User asks to "run A/B test", "evaluate harness", "run smoke test", or "benchmark all" - Pre-release verification to confirm zero regressions and 100% guard interception rate - Comparing new model profiles (e.g., Claude 3.7 Sonnet, GLM-5, GPT-4o) on Bare vs Epic --- ## 2. Command Reference & Execution Modes | Command | Scope | What It Runs | | :--- | :--- | :--- | | **`/benchmark full`** (or `/benchmark all`) | **Full Suite (Recommended)** | 1) Ring 0 Guard 50 Challenge Suite<br>2) 5 Golden Set Tasks (Bare vs Epic A/B)<br>3) Master comparison report | | **`/benchmark smoke`** | 5 Golden Set Tasks | Runs A/B evaluation across `task1` ~ `task5` | | **`/benchmark task <name>`** | Single Task | Runs A/B evaluation on a specific task (e.g. `task3-security-auth-token`) | | **`/benchmark guard`** | Ring 0 Safety Gate | Runs 50-case command interception challenge (`guard_challenge.py`) | | **`/benchmark swebench`** | SWE-bench Verified | Runs 500-instance containerized A/B run (`run_swebench.sh`) | | **`/benchmark report`** | Reporting Only | Reads `DIRECTOR-REPORT.md` and provides executive insights | ---