prompt-engineer-toolkit

# Prompt Engineer Toolkit ## Overview Use this skill to move prompts from ad-hoc drafts to production assets with repeatable testing, versioning, and regression safety. It emphasizes measurable quality over intuition. Apply it when launching a new LLM feature that needs reliable outputs, when prompt quality degrades after model or instruction changes, when multiple team members edit prompts and need history/diffs, when you need evidence-based prompt choice for production rollout, or when you want consistent prompt governance across environments. ## Core Capabilities - A/B prompt evaluation against structured test cases - Quantitative scoring for adherence, relevance, and safety checks - Prompt version tracking with immutable history and changelog - Prompt diffs to review behavior-impacting edits - Reusable prompt templates and selection guidance - Regression-friendly workflows for model/prompt updates ## Key Workflows ### 1. Run Prompt A/B Test Prepare JSON test cases and run: ```bash python3 scripts/prompt_tester.py \ --prompt-a-file prompts/a.txt \ --prompt-b-file prompts/b.txt \ --cases-file testcases.json \ --runner-cmd 'my-llm-cli --prompt {prompt} --input {input}' \ --format text ``` Input can also come from stdin/`--input` JSON payload. ### 2. Choose Winner With Evidence The tester scores outputs per case and aggregates: - expected content coverage - forbidden content violations - regex/format compliance - output length sanity Use the higher-scor...