ai-system-reviewlisted

# AI System Review Evaluate LLM-powered application code for reliability, quality, and cost. Covers context assembly, retrieval pipelines (RAG), prompt design, output validation, error handling, and spend controls. ## Usage ``` /ai-system-review # review staged changes /ai-system-review <path> # review a specific directory or file /ai-system-review --prompts # focus only on prompt quality /ai-system-review --costs # focus only on token cost controls ``` ## Behavior ### Step 1 — Map the LLM system boundary ```bash # Find model client calls grep -rn "openai\|anthropic\|bedrock\|vertexai\|litellm\|langchain\|llamaindex" \ --include="*.ts" --include="*.py" --include="*.js" -l . | head -20 # Find prompt templates find . -name "*.txt" -o -name "*.md" -o -name "*.jinja" | xargs grep -l "{{.*}}\|\${.*}\|<user>\|<system>" 2>/dev/null | head -10 ``` ### Step 2 — Review context assembly Check how context is built before each LLM call: | Concern | What to look for | Risk | |---|---|---| | Context window overflow | No token counting before call | Truncation silently corrupts prompt | | Injection risk | User input concatenated directly into system prompt | Prompt injection | | Stale context | No timestamp or recency check on retrieved chunks | Outdated information presented as current | | Missing metadata | Retrieved chunks lack source / date | Hallucination is unverifiable | | Token waste | Entire documents passed when only sections neede