vlm-ocr-evaluationlisted

# VLM-OCR Evaluation: Comparing OCR Systems Before You Commit ## Instructions Before running any OCR model across a whole corpus, run a controlled comparison on a small, human-transcribed sample and let the measured error rates pick the model. This skill is the **selection gate** that precedes the `vlm-ocr-pipeline` skill: use this to choose a model and document why, then use `vlm-ocr-pipeline` to run the chosen model at scale and `post-ocr-cleanup` to clean its output. For the hardest pages, where no single model is reliable, the multi-model voting logic in `model-council-voting` can be applied to OCR transcriptions as well. ### 1. Run a Comparison Before You Commit - Treat model choice as an empirical question, not a default — test, do not guess. A model that tops a general vision-language leaderboard, or that read one language well, can still fail on your script, era, or page layout. The only trustworthy signal is its error rate on pages like yours. - The comparison is cheap insurance. Evaluating a 50–100 page sample once prevents discovering, after a 50,000-page bulk run, that the chosen model silently dropped every table or collapsed on degraded scans. - Scope the comparison to the decision. A worked comparison of this kind runs nine systems on 64 pages spanning two languages and seven decades before committing to a bulk pipeline — enough to rank the candidates on the strata that matter, not a full corpus. - The output is a go/no-go gate: keep the model whose accurac