← ClaudeAtlas

scenariolisted

Author and manage holdout scenarios for behavioral validation. Scenarios are stored in .agents/holdout/ where implementing agents cannot see them. Triggers: "$scenario", "holdout", "behavioral scenario", "create scenario", "list scenarios".
boshu2/agentops · ★ 369 · AI & Automation · score 78
Install: claude install-skill boshu2/agentops
# Scenario Skill Author and manage holdout scenarios for Stage 4 behavioral validation. Scenarios are **holdout** — implementing agents cannot see them (enforced by hook). Evaluator agents validate code against scenarios during STEP 1.8 in `$validation`. ## Execution Steps ### Step 1: Initialize ```bash ao scenario init # Creates .agents/holdout/ with README ``` ### Step 2: Author Scenario Write a scenario JSON to `.agents/holdout/<id>.json` following `schemas/scenario.v1.schema.json`: ```json { "id": "s-2026-04-05-001", "version": 1, "date": "2026-04-05", "goal": "User can authenticate with valid credentials", "narrative": "A user visits login, enters valid credentials, expects dashboard redirect.", "expected_outcome": "Dashboard loads, session cookie is HttpOnly and Secure.", "acceptance_vectors": [ {"dimension": "correctness", "threshold": 0.9, "check": "grep -q 'HttpOnly' headers"}, {"dimension": "performance", "threshold": 0.7} ], "satisfaction_threshold": 0.8, "scope": { "files": ["src/auth/middleware.go"], "functions": ["Authenticate"], "behaviors": ["login flow"] }, "source": "human", "status": "active" } ``` ### Step 3: Validate ```bash ao scenario validate # Checks all scenarios against schema ``` ### Step 4: List ```bash ao scenario list # All scenarios ao scenario list --status active # Active only ``` ## Key Rules - Scenarios use **satis