evaluate
FeaturedEvaluates RAG retrieval and LLM-as-judge metrics (faithfulness, relevancy, context precision). Triggers: measure RAG quality, knowledge gap, RAG eval, golden dataset.
Install
Quality Score: 93/100
Skill Content
Details
- Author
- softspark
- Repository
- softspark/ai-toolkit
- Created
- 4 months ago
- Last Updated
- yesterday
- Language
- Python
- License
- Apache-2.0
Integrates with
Similar Skills
Semantically similar based on skill content — not just same category
monitor-rag-quality
Use this to measure and monitor the quality of a RAG (retrieval-augmented generation) pipeline - whether it retrieves the right context and answers faithfully. Trigger on "my RAG gives wrong answers", "is my retrieval any good", "the chatbot makes things up", "evaluate my RAG", "improve RAG accuracy". Diagnose whether the failure is in retrieval or generation - they need different fixes.
rag-evaluation
Measure retrieval and generation separately against a judged set, so you know whether a wrong answer came from the search or the model. Use when a RAG system is unreliable and every fix is a guess.
add-llm-evals
Use this when adding evaluation to an LLM/agent app - measuring output quality (correctness, faithfulness, relevance, safety) rather than just watching traces. Trigger on "add evals", "test my prompt", "is my RAG accurate", "catch regressions", "score outputs", or setting up an eval suite in CI. Covers offline (CI) and online (production LLM-as-a-judge) evaluation.