model-evaluation

# Model-Evaluation Skill ## Purpose This skill makes a medical-imaging model's **held-out evaluation task-correct and honest**: the right metric for the task and the prevalence, with uncertainty, calibration, and subgroup performance. It emits a **per-case metric table** that the publication statistics build on, and gates the metric choice against Metrics Reloaded (Maier-Hein & Reinke et al., *Nat Methods* 2024) and CLAIM 2024. It sits between `/model-validation` (which audits the split / design) and `/analyze-stats` (which owns the comparative inference). It computes the imaging-specific per-case metrics (surface distances, FROC, ECE of a softmax head); `/analyze-stats` owns DeLong / NRI / IDI / decision curves / MRMC. Like `/analyze-stats`, it **generates and executes** code on your predictions — numbers are never hand-typed. ## When to use - You have held-out predictions + ground truth and need task-correct metrics with CIs, calibration, and subgroup slices, plus a per-case table for the manuscript statistics. ## When NOT to use - Auditing the validation design / leakage → `/model-validation`. - DeLong / NRI / IDI / decision curves / MRMC reader study → `/analyze-stats`. - Building / training the model → `/model-scaffold`; LLM / MLLM → `/mllm-eval`. - Figure rendering → `/make-figures`. ## Workflow ### Phase 1 — Fix the analysis unit and the task State the task (segmentation / classification / detection / interactive / generative) and the **analysis unit** the met...