sglang-hicache

# SGLang HiCache — hierarchical KV cache Target audience: operators running SGLang on H100/H200/H20/B200-class datacenter GPUs in production, especially long-context agentic workloads (coding agents, RAG, multi-turn chat) with strong prefix locality. ## Why this matters Long-context inference is almost always KV-cache bound, not compute bound. HiCache extends effective KV capacity beyond HBM through pinned host DRAM (L2) and an optional distributed L3 (Mooncake / 3FS / NIXL / AIBrix / EIC / SiMM / file). Reported gains: TTFT –56% / throughput ×2 (Novita Qwen3-Coder-480B + 3FS), TTFT –84% on cache hits (Ant Group DeepSeek-R1-671B + Mooncake) — see [LMSYS blog 2025-09-10](https://www.lmsys.org/blog/2025-09-10-sglang-hicache/). LMSYS's headline "up to 6× / up to 80%" is uncited — trust the deployment-specific numbers. **Why this skill exists alongside `vllm-caching`:** the two stacks are now at parity on hybrid-attention models, so the choice is about topology and L3 ecosystem rather than a capability gap. **The old "vLLM is broken on 2026 hybrids" framing is stale** — vLLM's native offload gained HMA support in v0.21.0 and HMA became the default for `SupportsHMA` connectors in v0.23.0, and LMCache MP added hybrid support in its 0.5 line. SGLang HiCache reached hybrid support first (v0.5.11 SWA + 3FS Mamba/DSA, default-on via UnifiedTree in v0.5.13) and still has the broader menu of distributed L3 backends. Where vLLM's in-process `LMCacheConnectorV1` remains genuinely block...