← ClaudeAtlas

eval-setlisted

Build and run a project-specific retrieval eval so changes to your rules or base prompt are scored, not eyeballed. Mirrors tests/ground_truth.json + `clawness eval`: you write prompt→expected-rule cases, then measure MRR@k and hit-rate before and after an edit. Run it after trimming a base prompt into ranked retrieval (see openclaw-audit / claude-md), or whenever you change rules and want proof retrieval still surfaces what matters.
fullymiddleaged/Clawness · ★ 4 · AI & Automation · score 77
Install: claude install-skill fullymiddleaged/Clawness
# Score your retrieval, don't eyeball it When you move guidance out of an always-loaded base prompt (a bloated `CLAUDE.md`, an OpenClaw `SOUL.md`/`AGENTS.md`) into Clawness's ranked retrieval, you trade a guarantee for a probability: the content used to be present on *every* turn; now it surfaces only when the prompt is relevant enough to rank it. That trade is usually right — it is the whole point of `/clawness:claude-md` and `/clawness:openclaw-audit` — but it is only safe if you can *check* that the content still surfaces for the prompts that need it. This skill builds that check. It is the same machinery Clawness gates its own corpus with: a labelled set of `prompt → expected rule ID(s)` cases, scored by MRR@k and hit-rate via `clawness eval`. The output is a number that moves when retrieval regresses, so a rule edit that quietly stops surfacing shows up instead of hiding until someone hits it in anger. This is harness-agnostic — it evaluates Clawness retrieval, which is identical under Claude Code and under the OpenClaw adapter. Nothing here touches OpenClaw's own prompt. ## When to run it - **After trimming a base prompt** into `.clawness/rules/` — write a case for each thing the moved content used to guarantee, then confirm hit-rate is 1.0 before you delete the original. This is the verification step `openclaw-audit`/`claude-md` point at. - **After editing or adding rules** — re-run to confirm you didn't push an existing rule out of the top-k for prompts tha