ai-llm-inference

# LLM Inference Production Skill Hub Operational guidance for choosing and tuning modern inference stacks. Focus on runtime fit, routing, output guarantees, adapter loading, multimodal serving, and measured performance under load. Use current primary sources for volatile facts such as versions, hardware support, benchmarks, pricing, and release status. ## ASCII Flow ```text serving workload | v intake model + modality + context + QPS + latency SLO + hardware + output contract | v serving design engine + router + batching + cache + quantization + structured outputs | v benchmark TTFT + ITL + throughput + error rate + quality floor | v production serving path capacity plan + rollout + monitoring + rollback thresholds ``` ## When to Use This Skill Use this skill when the user asks for: - inference engine selection or stack comparison - latency, TTFT, ITL, or throughput optimization - cache-aware routing or control-plane design - quantization strategy by runtime and hardware - multi-GPU or multi-node serving - structured outputs or constrained decoding at serve time - multimodal or encoder-decoder serving patterns - LoRA or multi-adapter serving - cost reduction for self-hosted or API inference - benchmarking, profiling, or capacity planning - CPU or edge deployment with GGUF or llama.cpp, or on-device NPU (per-tensor scales, static shapes) ## Scope Boundaries - Prompting, fine-tuning, eval sets -> [ai-llm](../ai-llm/SKILL.md) - RAG pipeline desig...