gpu-host-tuning

Solid

Audit AND tune Linux/GPU inference hosts — read-only host snapshot (CPU power state, C-states, NUMA topology, PCIe link state, GPU settings, kernel boot params, sysctl, ulimits, IRQ affinity, container runtime), optional pinned-host↔GPU memcpy bench (torch + numactl), and per-lever cheat-sheets to flip settings (governor, EPP, cpuidle, persistence, ECC, hugepages, intel_iommu, NCCL env, tuned-adm profiles, Dell/Supermicro/HPE BIOS guidance). Sits beneath any inference framework (vLLM, sglang, TensorRT-LLM) — about the host, not the framework.

AI & Automation 3 stars 1 forks Updated yesterday MIT

Install

View on GitHub

Quality Score: 79/100

Stars 20%
20
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
80
License 10%
100
Description 5%
100

Skill Content

# gpu-host-tuning Host-side tuning + audit for Linux GPU inference servers. Sits *beneath* any inference framework (vLLM, sglang, TensorRT-LLM, llama.cpp). **Assumes a working host.** Driver, `nvidia-fabricmanager`, and NVLink/NVSwitch must already be healthy — if `nvidia-smi` doesn't list every GPU, or fabricmanager isn't `active (running)`, that is a bring-up problem, not a tuning one: use `nvidia-datacenter-bringup` (same `inference-host` plugin) first. Tuning a half-initialised host produces numbers that describe the fault, not the hardware. Three modes: 1. **Audit** — read-only snapshot 2. **Bench** — ground-truth pinned-host↔GPU memcpy ceiling 3. **Tune** — apply individual levers from the cheat-sheet This file is a pointer map. The actual logic lives in `scripts/` and the authoritative references in `references/`. ## Quick start ```bash # From the skill directory — typically ~/.claude/skills/gpu-host-tuning # (personal) or .claude/skills/gpu-host-tuning (project install). # Audit (read-only, ~60s) ./scripts/collect.sh # Audit + pinned-memcpy bench (needs torch + CUDA, ~5 min) ./scripts/collect.sh --bench ``` The script prompts for the output parent dir on first interactive run and remembers the choice. Override via `--out <dir>` or `HOST_AUDIT_DIR=<dir>`. Default snapshot dirname is `gpu-host-tuning-<host>-<UTC>`. ## What the snapshot captures One file per probe, numbered by section. See [`references/probe-interpretation.md`](references/probe-interpretation...

Details

Author
air-gapped
Repository
air-gapped/skills
Created
3 months ago
Last Updated
yesterday
Language
Python
License
MIT

Integrates with

Bundled in these plugins

Similar Skills

Semantically similar based on skill content — not just same category

Data & Documents Listed

asus-esc8000-gpu-bios-tuning

Apply a low-latency GPU-serving + VM-passthrough BIOS profile to ASUS ESC8000-E12P (ASMB12/AMI-BMC, Xeon 6/GNR) servers over Redfish — diff current settings, stage, reboot, verify. Use when asked to optimize/tune BIOS for GPU inference/serving, set performance power profile, enable Resize BAR / SR-IOV / disable SNC, or replicate one server's BIOS config onto another ESC8000.

3 Updated 1 weeks ago
soulmachine
AI & Automation Solid

vllm-performance-tuning

vLLM performance-tuning operator reference — tuning workflow (baseline → bottleneck → knob → re-bench), fused-MoE kernel autotune (`benchmark_moe.py` generates `E=N,N=M,device_name=X.json` configs), DeepEP all-to-all + expert parallelism + EPLB, CUDA graph modes (FULL_AND_PIECEWISE default), torch.compile AOT + compile cache, scheduler knobs (`--max-num-batched-tokens`, `--max-num-seqs`, `--async-scheduling`), TP/EP/DP/PP decision tree, NCCL/DCGM on H100/H200/B200/GB200, PD disaggregation (Nixl/Mooncake/LMCache), known regressions + vendor quirks (v0.14→0.15.1 MiniMax, MI300X FP8<BF16, DeepGEMM M<128 TTFT).

3 Updated yesterday
air-gapped
AI & Automation Solid

nvidia-datacenter-bringup

Bring up NVIDIA HGX/DGX datacenter GPU hosts on Ubuntu 24.04 LTS — air-gapped or connected, Secure Boot enabled. Covers B300/B200/H100/A100/L40S/L4 driver+fabricmanager+NVLSM+DOCA-OFED install order and exact package set from NVIDIA CUDA repo + DOCA repo. Triggers on B300/B200/HGX/DGX install, "fabricmanager won't start", "system not yet initialized" / cudaErrorSystemNotReady, NVLSM missing, ib_umad not loading, DOCA-OFED before NVIDIA driver, nvidia-driver-pinning-XXX, nvlink5-XXX, nvidia-open vs cuda-drivers, "Blackwell requires open kernel modules", ConnectX-7/8 bridge device, FM exact-version-match, gpu-operator cuda-validator CrashLoopBackOff, B300 PCI ID 0x3182, air-gap CUDA + DOCA mirror, three-tier DOCA GPG key, MOK enrollment, DKMS sign, Dell PowerEdge XE9780/XE9785 baseboard firmware v1.4.30, iDRAC Redfish virtual AC cycle DellOemChassis.ExtendedReset, generic "install nvidia driver ubuntu 24.04 datacenter".

3 Updated yesterday
air-gapped