orchestrate-escalation-designlisted

# Orchestrate: Escalation Design **Direct evidence**: the support-agent challenge's starter repo states the hard requirement — *"Must escalate high-risk, sensitive, or unsupported cases"* instead of guessing. HackerRank's own description of the dataset confirms it's built with edge cases, prompt injection attempts, and jailbreaking tests specifically so that **escalating everything and replying to everything both result in failure** — meaning the graded signal lives almost entirely in a calibrated middle ground, not at either extreme. ## Why this can't be an afterthought If escalation is implemented as "if confidence < threshold, escalate" bolted onto an otherwise-complete classify-and-respond pipeline, the threshold gets tuned by guesswork against however the demo happens to run — not against a considered model of *what kinds of cases actually warrant escalation*. The published dataset is specifically designed to punish that approach from both directions: too eager to escalate loses points for cases that should have been resolved automatically; too eager to respond loses points (and, in the adversarial cases, potentially demonstrates a real safety failure) for cases that needed a human. ## What "escalate deliberately" looks like as a design Treat escalation as a genuine decision with named categories, not a catch-all: - **High-risk**: cases where an incorrect automated response has real consequences (security, financial, legal-adjacent content) - **Sensitive**: cases t