orchestrate-multi-strategy-evaluationlisted

# Orchestrate: Multi-Strategy Evaluation **Direct evidence**: the multi-modal-review (June) challenge's evaluation criteria explicitly require *"comparison of ≥2 strategies/prompts/configurations"* and *"final approach documentation"* as mandatory analysis — not optional polish. This is a formalized, graded version of ordinary good engineering practice: don't ship your first idea without checking whether a second one does better. ## What this looks like in practice 1. **Build against the sample/dev dataset, not the golden dataset you don't have.** Every Orchestrate challenge ships a `sample_*.csv` with known expected outputs specifically for this purpose. 2. **Implement at least two genuinely different approaches** to some meaningful part of the system — not two trivial variations. Examples: a single-call classification prompt vs. a two-step "extract evidence, then classify" pipeline; keyword-based corpus retrieval vs. embedding-based retrieval; a strict rule-based escalation policy vs. a model-judged one. 3. **Score both against the sample set** using the same metric (accuracy against known labels, or a proxy metric if labels are qualitative) and record the numbers, not just an impression. 4. **Document why you chose what you chose** — including what the losing approach got wrong, specifically. "Approach B mis-classified 3 of 20 sample tickets because it conflated `bug` and `product_issue` when a ticket mentioned an error message without describing a workflow" is evidence