gpu-optimizer

Featured

GPU optimization for consumer NVIDIA GPUs (8-24GB VRAM) covering mixed precision, gradient checkpointing, XGBoost GPU, CuPy/cuDF migration, and torch.compile. Triggers on: "optimize GPU training", "speed up CUDA", "reduce OOM", "migrate NumPy to CuPy", "manage GPU memory", "benchmark PyTorch".

AI & Automation 316 stars 47 forks Updated yesterday MIT

Install

View on GitHub

Quality Score: 93/100

Stars 20%
83
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# GPU Optimizer Expert GPU optimization for consumer GPUs with 8–24GB VRAM. Evidence-based patterns only. ## Hardware Profile Fill in your hardware before applying optimizations: | Property | Your Value | | ----------------- | ------------------------------------------------ | | GPU model | (e.g., RTX 4080 Mobile, RTX 3090, RTX 4090) | | VRAM | (e.g., 12GB, 16GB, 24GB) | | CUDA version | (`nvidia-smi` → top-right) | | TDP / power limit | (laptop vs desktop affects sustained throughput) | | Driver version | (`nvidia-smi` → top-left) | Key constraint: VRAM capacity determines which strategies apply. Patterns below are annotated with minimum VRAM requirements where relevant. ## Optimization Categories ### 1. XGBoost GPU Acceleration **DMatrix vs QuantileDMatrix:** ```python # GPU-optimized: QuantileDMatrix is 1.8x faster dtrain = xgb.QuantileDMatrix(X_train.astype(np.float32)) dval = xgb.QuantileDMatrix(X_val.astype(np.float32)) # Standard: DMatrix (use for inference only) dtest = xgb.DMatrix(X_test.astype(np.float32)) ``` **Critical Parameters:** ```python params = { 'tree_method': 'hist', # GPU-accelerated histogram 'device': 'cuda:0', # Explicit GPU device 'max_bin': 256, # Higher bins = better splits (VRAM permitting) 'grow_policy': 'depthwise', # vs 'lossguide' for i...

Details

Author
Mathews-Tom
Repository
Mathews-Tom/armory
Created
6 months ago
Last Updated
yesterday
Language
Python
License
MIT

Bundled in these plugins

Similar Skills

Semantically similar based on skill content — not just same category

AI & Automation Featured

ai-distributed-training

Guides multi-GPU pre-training: DDP, FSDP2, ZeRO, tensor/pipeline/expert parallelism, fp8/Muon. Use when scaling a run, training MoE, or reproducing GPT-2 on rented GPUs.

87 Updated 1 weeks ago
vasilyu1983
AI & Automation Listed

vllm-performance-tuning

vLLM performance-tuning operator reference — tuning workflow (baseline → bottleneck → knob → re-bench), fused-MoE kernel autotune (`benchmark_moe.py` generates `E=N,N=M,device_name=X.json` configs), DeepEP all-to-all + expert parallelism + EPLB, CUDA graph modes (FULL_AND_PIECEWISE default), torch.compile AOT + compile cache, scheduler knobs (`--max-num-batched-tokens`, `--max-num-seqs`, `--async-scheduling`), TP/EP/DP/PP decision tree, NCCL/DCGM on H100/H200/B200/GB200, PD disaggregation (Nixl/Mooncake/LMCache), known regressions + vendor quirks (v0.14→0.15.1 MiniMax, MI300X FP8<BF16, DeepGEMM M<128 TTFT).

5 Updated 1 weeks ago
air-gapped
AI & Automation Listed

ai-hardware-selection

Selecting accelerators for AI workloads: GPU vs TPU vs NPU vs FPGA vs CPU, and the metrics that actually decide it — memory capacity & bandwidth, TOPS/ FLOPS, interconnect, and cost/Watt. Architect-level hardware-fit reasoning. USE WHEN: choosing AI hardware/accelerators, "which GPU", "TPU vs GPU", "NPU", "FPGA", "HBM/memory bandwidth", "TOPS", "cost per token", VRAM sizing for a model, training vs inference hardware, accelerator interconnect. DO NOT USE FOR: serving software topology (use `inference-serving-topology`); on-device runtimes (use `edge-inference`); generic CPU perf (use systems/hardware-aware-design).

33 Updated today
claude-dev-suite