litellm-valkeylisted

Run LiteLLM proxy multi-pod with Redis/Valkey (standalone, Sentinel, or Cluster) so rate limits, budgets, and router state are actually shared across pods. Centerpiece: LiteLLM's dominant failure mode is SILENT fallback from Redis to per-pod in-memory enforcement (≥6 independent code paths, log-level warning, no metric) — a fleet quietly enforces N× its configured limits. Covers the 4-way coordination-Redis resolution order (incl. the undocumented `general_settings.coordination_redis` block, first-class since v1.93.0, and the DB-persisted UI value that silently outranks the config file), Sentinel/Cluster support per subsystem with fix versions, the per-pod-vs-shared truth table with real staleness bounds, documented knobs that are no-ops, and the open-issue catalog (counter drift, connection churn, ssl_check_hostname).
air-gapped/skills · ★ 5 · AI & Automation · score 78

Install: claude install-skill air-gapped/skills

# LiteLLM proxy + Redis/Valkey multi-pod — operator reference Target: operating LiteLLM proxy with 2+ replicas (typically the in-repo Helm chart) where Redis or Valkey is supposed to make rate limits, spend/budget enforcement, cooldowns, and locks fleet-wide. Grounded in source at `4d543245` (v1.95.0-dev, 2026-07-29; latest stable v1.94.0) plus a GitHub-issue sweep of the same date. LiteLLM releases weekly and fixes land fast — treat every claim as version-stamped, and re-verify on the deployed tag. Sibling skill: the proxy's management REST API (keys, teams, budgets semantics) is **`litellm-api`**. Migrating the Redis itself to Valkey is **`redis-to-valkey`**. The single most important thing to internalize: **when Redis fails — or is never wired in — LiteLLM does not fail. It silently enforces everything per-pod.** No 5xx, no metric, log-level `warning` at best. A fleet of N pods enforces N× every rate limit and lets budgets drift. Verifying that coordination is *actually* shared is the operator's job; nothing in the product surfaces the loss. Read `references/silent-degradation.md` first. ## Which Redis am I actually coordinating through? The coordination Redis (rate limits, spend counters, pod locks — NOT the response cache) is resolved in this precedence order (`proxy_server.py` `_init_coordination_redis` → `_init_cache`): 1. **`general_settings.coordination_redis`** in config.yaml — the only clean way to point coordination at its own Redis. First-class since **v1.9