ai-evals-benchmark-expert
SolidExpert guide for LLM & AI Agent automated evaluations (Evals), Promptfoo, DeepEval, Ragas, regression benchmarking, and deterministic assertions / Panduan ahli evaluasi otomatis LLM & AI Agent (Evals), benchmarking regresi, dan pengujian kualitas.
Install
Quality Score: 87/100
Skill Content
Details
- Author
- roedyrustam
- Repository
- roedyrustam/vibes-plug
- Created
- 3 months ago
- Last Updated
- today
- Language
- Python
- License
- MIT
Integrates with
Similar Skills
Semantically similar based on skill content — not just same category
ai-evals
Use when testing an LLM-backed feature, prompt, tool loop or multi-step agent, where the same input can produce different outputs and a prompt or model change can regress behaviour with no code diff. The behaviour spec that precedes the prompt, scenario datasets including adversarial and degradation classes, deterministic assertions over OpenTelemetry traces, calibrated LLM-as-judge, CI gates with baselines, human-in-the-loop, and the production scoring loop that turns incidents into scenarios.
llm-eval-testing
When the user wants to design, build, or operate evaluations (evals) for LLM-powered products — chatbots, RAG systems, agents, classification, summarization, structured output. Use when the user mentions "LLM evals," "evals," "RAG evaluation," "RAGAS," "DeepEval," "LangSmith," "LangFuse," "PromptLayer," "OpenAI evals," "judge model," "rubric eval," "LLM-as-judge," "Inspect AI," "AnthropicEvals," "Vertex evals," "Braintrust," or "regression tests for prompts." For AI testing tools see ai-augmented-testing. For chaos see chaos-engineering. For production monitoring see production-testing.
ai-evals
Designs trustworthy LLM, agent, responsible-AI, and multimodal evaluations. Use when measuring quality, fairness, privacy, grounding, safety, or judge reliability.