agent-trajectory-evaluatorlisted

Final-answer evals tell you the agent failed; they don't tell you *where*. An agent that returns the right number might have called the wrong tool first, looped on a flaky API, or stumbled into the answer through a path that collapses on the next input. This skill makes the agent's **process** inspectable: capture the full trajectory — every decision, tool call, argument, and result — then score it on the axes that actually predict failure, asserting what's checkable and judging only what isn't. ## When to use this skill - You're building or debugging a tool-using / multi-step agent and a final-answer eval says "wrong" without saying why. - A prompt or model change kept the answers correct but you suspect the agent got slower, looped more, or recovers worse — and you need to prove it. - You're adding a new tool and want to confirm the agent selects it correctly instead of brute-forcing with the old one. - Failures are intermittent and you can't tell whether the agent is fragile (lucky path) or robust (sound path). ## Instructions 1. **Capture the full trajectory as a structured, replayable log — one record per step.** Final-answer-only logging is the root cause of un-diagnosable failures. Each step records: the model's decision (the assistant turn, including thinking-block summaries if present), the tool called and its exact arguments, the raw tool result (success/error), and any externalized state (files written, working dir, retry count). Use a stable schema so two runs