failure-to-eval-capturelisted
Install: claude install-skill Abhillashjadhav/PM-agent-OS
# Failure-to-Eval Capture
Every production failure becomes a permanent test — scrubbed of the people it happened to, but still carrying the trap it fell into. Scrub the identity, keep the mechanism.
## Verification gates (defined first; output is blocked until all pass)
- **G1 — Scrubbed:** all PII and identifying detail (names, emails, orgs, distinctive figures) replaced with stable placeholders; a scrub table shows placeholder classes, never relisting the original values; the unscrubbed failure never appears in the eval artifact.
- **G2 — Pattern demonstrably preserved:** the case states the failure mechanism and shows that the scrubbed input retains its trigger structure — an explicit preservation argument ("mechanism: entity-invention from org-name context; scrubbed input keeps org-as-topic + attendee-owned action item"). Over-scrubbing that deletes the trigger fails this half as hard as leaking PII fails the other.
- **G3 — Mechanically assertable:** the expected-behavior assertion is checkable without a judge wherever possible (every attendee in output ∈ input attendee list); the case carries id, failure class, provenance (date/ticket, never customer identity), and its regression wiring.
## Steps
1. **Name the mechanism first.** What made the output wrong — not "bad summary" but "invented an entity from an org name mentioned as topic, then reassigned an action item to it". The scrub is designed around protecting this mechanism; naming it first is what makes G2 chec