llm-eval-harnesslisted

# llm-eval-harness Builds the measurement that turns "the new prompt feels better" into "the new prompt scores 0.91 vs 0.84 baseline, with zero regressions on the critical subset." Without it, every prompt or model change is a vibe with a deploy button. Provider-neutral by design — the harness shape is the same whether the feature runs on Claude, GPT, Gemini, or a local model; only the runner differs. Distinct from `ai-usage-report` (telemetry after the fact) and `success-metrics` (product outcomes) — this is the **dev-loop quality gate**. ## How to respond 1. **Pin the task and the unit of judgment.** What does the feature do (classify / extract / summarize / answer-with-RAG / agentic-multi-step), and **what does one gradeable output look like**? Ask at most 2 questions, spent on the failure that hurts most ("what's a wrong answer that would actually cause a problem?") and whether ground truth exists. The answer-that-hurts shapes the adversarial cases and the critical subset. 2. **Build the golden set with three deliberate buckets** (case format in [`templates/`](templates/)): - **Representative** — the real distribution of inputs, sampled from production/logs where possible, not invented. This sets the headline number. - **Adversarial / edge** — the inputs that break things: ambiguous, out-of-scope, prompt-injection attempts, empty/malformed, the long tail. This is where models actually differ. - **Regression** — every past failure, frozen as a case the moment