← ClaudeAtlas

agent-harness-evaluationlisted

Evaluate agent harness reliability through replay, restart, cancellation, runtime reconfiguration, concurrent generations, rollback, context pressure, and release gates. Excludes generic tests, surveys, and prompt/model-only benchmarks.
Xopoko/plug-n-skills · ★ 11 · AI & Automation · score 77
Install: claude install-skill Xopoko/plug-n-skills
# Agent Harness Evaluation Bundled commands use `$PLUGIN_ROOT` (`$env:PLUGIN_ROOT` in PowerShell; same path suffix) for the plugin root. Set it once: use the host's plugin-root variable when defined (Claude Code: `PLUGIN_ROOT="$CLAUDE_PLUGIN_ROOT"`), otherwise the absolute path of this skill folder's `../..`. Evaluate the behavior of the complete agent system, not just the final model answer. Use deterministic post-state and trace evidence wherever possible; use judgment only for residual qualitative questions. Read `$PLUGIN_ROOT/references/agent-harness-evaluation.md` for suite and reporting guidance and `$PLUGIN_ROOT/references/agent-harness-contracts.md` for artifact, state, and event contracts. Use `$PLUGIN_ROOT/references/agent-harness-landscape.md` only when comparing systems or provider capability surfaces. ## Evidence Boundaries - Model output is not proof that a side effect was authorized, executed, durable, or correct. - A transcript is not a checkpoint and cannot prove restart or recovery behavior. - A diagnostic prompt render does not prove the exact prompt, skill catalog, or tool schemas seen by another task or runtime origin. Bind the exact rollout and runtime generation, and capture separately supplied tool schemas independently. - Prompt instructions, skills, and scanners do not prove sandbox or policy enforcement. - Do not infer exactly-once behavior from a clean run. Test duplicate delivery, idempotency, reconciliation, and partial effects. - Comp