model-upgrade-evaluatorlisted
Install: claude install-skill Abhillashjadhav/PM-agent-OS
# Model Upgrade Evaluator
Release notes are hypotheses. A new model changes two lists — what might now break (current prompts) and what might now work (shelved ideas) — and both get answered by runs, not by the announcement.
## Verification gates (defined first; output is blocked until all pass)
- **G1 — Every verdict cites a run:** MIGRATE/STAY/RESURRECT/STILL-DEAD verdicts exist only with per-case run evidence (counts, deltas, the failing or passing cases named). Before runs execute, every verdict slot reads `PENDING`. Release-note claims appear only as hypotheses-to-test, never as capability facts.
- **G2 — Kill reasons become resurrection tests:** each shelved idea is re-tested against the specific reason it was killed (the historical failing case or its closest reconstruction) — not a fresh demo that avoids the old trap. A resurrection test that doesn't encode the kill reason fails.
- **G3 — Budget-honest coverage:** the plan ranks runs within the stated budget, names what's cut, and marks unscheduled items `UNTESTED — not a verdict`. Claimed coverage beyond scheduled runs fails.
## Steps
1. **Inventory both lists:** production prompts with their golden sets (these re-run under regression-gatekeeper rules — fail-class assertions must hold), and the shelved-ideas log with each idea's kill reason stated as a testable condition.
2. **Convert release-note claims into targeted hypotheses.** "Better instruction-following" → which of our golden failures were instruction-fo