prompt-regression

Solid

Detects harness regressions by running standard prompt probes after rule/skill changes and comparing outputs against saved baselines. Triggers on "prompt regression", "did my changes break anything", "regression check", "test harness changes".

AI & Automation 8 stars 0 forks Updated today MIT

Install

View on GitHub

Quality Score: 79/100

Stars 20%
32
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# prompt-regression — Harness Regression Detection After CLAUDE.md edits, rule changes, or new skill additions, harness behavior can silently regress. This skill runs a lightweight probe suite against the changed assets and compares outputs against saved baselines to surface regressions before they reach production. > **Scope distinction** > - harness-doctor: structural completeness (files, links, drift) > - prompt-regression: **behavioral correctness** — did the change alter expected AI response patterns? --- ## Triggers - `/prompt-regression` - "prompt regression", "regression check", "regression test" - "did my rule change break anything", "test harness changes", "verify harness behavior", "make sure my edit didn't change behavior" - After significant CLAUDE.md edits or new skill commits --- ## Execution Steps ### Step 1. Identify Changed Assets ```bash # What changed since last commit (or last N commits) git diff HEAD~1 --name-only -- CLAUDE.md .claude/ plugins/ ``` Classify each changed file: - `CLAUDE.md` → **core behavior** (high impact) - `.claude/rules/*.md` → **rule layer** (medium impact) - `plugins/*/skills/*/SKILL.md` → **skill behavior** (scoped impact) - `plugins/*/skills/*/SKILL.md` (trigger phrases changed) → **trigger routing** (high impact) If no changes detected: report "No harness changes since last commit — regression check skipped." --- ### Step 2. Load Probe Suite Check for custom probes: ```bash ls .claude/regression/probes.md 2>/dev/nul...

Details

Author
chrono-meta
Repository
chrono-meta/forge-harness
Created
2 months ago
Last Updated
today
Language
Shell
License
MIT

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category