← ClaudeAtlas

adversarial-testinglisted

Use when authoring evals/predicates (Scout, TDD-Engineer), framing an audit (Auditor), or justifying which phases run (PhaseAdvisor/router). Codifies Google's 4-phase adversarial-testing methodology mapped onto evolve-loop's eval, audit, routing, and red-team surfaces.
mickeyyaya/evolve-loop · ★ 2 · Testing & QA · score 75
Install: claude install-skill mickeyyaya/evolve-loop
# Adversarial Testing for evolve-loop > Canonical methodology reference. Derived from Google's [Adversarial Testing for Generative AI](https://developers.google.com/machine-learning/guides/adv-testing), mapped onto this repo's existing inward-facing adversarial machinery (mutation testing, adversarial auditor, EGPS exit-code grounding). Referenced by: `agents/evolve-scout.md` (§eval integrity), `agents/evolve-tdd-engineer.md` (test diversity), `agents/evolve-auditor.md` (input categories), `go/internal/core/router_proposer.go` (`buildRoutingPrompt`), `acs/red-team/`. This file is the single source of truth — those consumers reference it, they do not re-derive it. ## Table of Contents 1. [Methodology overview (4 phases)](#1-methodology-overview) 2. [Phase 1 — Identify adversarial inputs](#2-phase-1--identify-adversarial-inputs) 3. [Phase 2 — Seed → synthesize → diversify](#3-phase-2--seed--synthesize--diversify) 4. [Phase 3 — Generate & annotate outputs](#4-phase-3--generate--annotate-outputs) 5. [Phase 4 — Report & mitigate](#5-phase-4--report--mitigate) 6. [Diversity checklist (eval authoring)](#6-diversity-checklist-eval-authoring) 7. [Phase-advisor rubric](#7-phase-advisor-rubric) 8. [Auditor framing extensions](#8-auditor-framing-extensions) 9. [Red-team predicate catalogue](#9-red-team-predicate-catalogue) 10. [Anti-gaming reference](#10-anti-gaming-reference) ## 1. Methodology overview Adversarial testing proactively tries to *break* a system with inputs most likel