eval
SolidUse this skill to run an honest session-process evaluation (Standard v1, aiat-llm-eval/1.0) — score the last completed orchestrator session against the pre-registered rubric-v1 dimensions, run /eval, evaluate this session, produce an eval report, or re-verify a stored eval run for reproducibility. Deterministic-first with an optional advisory LLM judge; never produces a global score.
Install
Quality Score: 89/100
Skill Content
Details
- Author
- Kanevry
- Repository
- Kanevry/session-orchestrator
- Created
- 3 months ago
- Last Updated
- today
- Language
- JavaScript
- License
- MIT
Integrates with
Bundled in these plugins
Similar Skills
Semantically similar based on skill content — not just same category
evaluating-skills
Use when testing whether a new skill improves agent behavior, or when validating a change to an existing skill's language.
advanced-evaluation
This skill should be used for advanced LLM evaluation: LLM-as-judge systems, direct scoring, pairwise comparison, rubric calibration, evaluator bias mitigation, confidence scoring, and automated quality assessment.
eval-writer
Authors rigorous eval suites for AI agents, skills, and LLM systems — grounded in the 2026 eval-writing consensus (trace-driven error analysis, binary LLM judges, cross-family validation, α/κ agreement). Produces characterization, failure taxonomies, judge prompts, rubrics, and calibration protocols that harnesses (pmo-skill-refiner, CI) then execute. Two modes — Author (write from scratch) and Review (audit against the framework). First-class playbooks for per-skill evals and for pipeline stage-gate judgment content; generic fallback for arbitrary AI systems. Use whenever the user asks to write evals, audit evals, add eval coverage, calibrate a judge, build a rubric, write a judge prompt, or diagnose why a judge keeps passing broken outputs.