← ClaudeAtlas

probelisted

Construct a novelty probe that distinguishes genuine situated reasoning from pattern-matching against a checklist. Build a pair of cases that look similar on the surface but differ in a material way; observe whether the agent's response diverges where it should. Measures Autonomous Reasoning Fidelity. USE WHEN: test reasoning quality, is the agent actually reasoning, distinguish reasoning from compliance, stress test, novelty injection, ARF measurement.
ntholm86/principles-of-earned-autonomy-skills-suite · ★ 0 · AI & Automation · score 71
Install: claude install-skill ntholm86/principles-of-earned-autonomy-skills-suite
# Probe *Build a situation the checklist couldn't anticipate. See whether reasoning emerges or pattern-matching is exposed.* *Memory Model role: Produces external ARF evidence — probe verdicts recorded in the trail are the primary signal that the loop is reasoning rather than pattern-matching.* This is the only skill in the suite that genuinely tests something external. Improve makes the agent better; Probe finds out whether it is reasoning at all. ## Governing principles Probe operationalizes **Autonomous Reasoning Fidelity (ARF)** — the external signal that an agent is genuinely reasoning rather than pattern-matching. In routine cases, the two produce identical-looking trails. The distinguishing evidence emerges only under structured novelty. Full statement of the principles: [PRINCIPLES.md](../PRINCIPLES.md) — read it if available, but this skill operates fully without it. ## The work ### 1. Identify the claim State precisely what claim about the agent's reasoning you intend to test. Examples: - "Improve's *Challenge the first read* step actually surfaces a real gap when one exists, not just performs skepticism." - "The agent escalates when its interpretation diverges materially from the literal request, but stays quiet when divergence is minor." - "The agent records `[!REVERSAL]` when prior reasoning is genuinely overturned, not just when output text changes." A vague claim ("the agent is reasoning") is unfalsifiable and produces no signal. Sharpen it until pas