add-llm-fallbackslisted

# Add LLM fallbacks and resilience LLM providers rate-limit, time out, 500, and occasionally go down. A single unguarded call means your app goes down with them. Add layered resilience, and instrument it so you can see when it kicks in. ## The layers (add all of them) 1. **Timeouts.** Every call needs a sane timeout. An unbounded call can hang a request forever. Set it based on your p95 latency plus headroom. 2. **Retries with backoff.** Retry transient failures (429, 500, 503, timeouts) with *exponential backoff + jitter*. Cap the number of retries and cap the total wait, and honor `Retry-After` when present. A naive retry loop turns a rate limit into a retry storm (and a big bill, see `reduce-llm-cost`). 3. **Do not retry non-transient errors.** 400s (bad request), auth errors, and content-policy refusals will fail again. Retrying them wastes time and money. 4. **Model / provider fallback.** If the primary model errors or is down, fall back to a secondary (same provider different model, or a different provider). A gateway (LiteLLM, Portkey) gives you this with config instead of code. 5. **Graceful degradation.** When everything fails, return a sensible fallback (cached answer, a "try again shortly" message, a simpler non-LLM path) instead of a stack trace. ## Instrument it (so resilience is not invisible) The failure mode of good resilience is that it hides problems. Emit telemetry so you still see them: - A span/event when a **retry** happens (and how many). - A span