Local LLM & GPU VRAM Compatibility Calculator (Ollama & Llama.cpp)
Local LLM & GPU VRAM Compatibility Calculator 2026 Models
Select your GPU VRAM (or Apple Silicon Unified Memory), system RAM, and target model to check exact memory footprint, KV cache overhead, and Ollama speed tier.
Top Recommended Models That Fit 100% on Your Current Hardware:
How Local LLM VRAM Math Works (Weights + KV Cache)
Running open-weight models locally with Ollama, LM Studio, or llama.cpp requires more memory than just the downloaded GGUF file size. Your GPU VRAM (or Apple Silicon Unified Memory) must hold three distinct blocks simultaneously:
- Model Weights: Determined by parameter count and quantization bit-depth. A
Q4_K_M4-bit quant uses roughly0.58 GBper billion parameters, whereasQ8_0doubles the memory footprint. - KV Context Cache: Every token in your conversation or RAG context window consumes extra VRAM. Increasing context from
8Kto64Kor128Ktokens can add 4 GB to 14 GB of extra VRAM demand on larger dense models. - OS & CUDA/Metal Overhead: Windows/Linux desktop compositors and CUDA runtime buffers reserve roughly
0.7 GB – 1.2 GBof VRAM before a model even loads.
Frequently Asked Questions
What happens if a model exceeds my GPU VRAM?
Ollama and llama.cpp automatically split layers between your GPU VRAM and system DDR4/DDR5 RAM. While the model will still run as long as your system RAM holds the overflow, token generation speed typically drops by 3x to 5x (from 40+ tok/s down to 6–12 tok/s).
Why are MoE (Mixture-of-Experts) models like Qwen3.6-35B-A3B faster?
Mixture-of-Experts models store all parameters in memory (e.g., 35B total), so they still require ~20 GB of VRAM to load, but only activate ~3B parameters per generated token—delivering the speed of a tiny 3B model with the reasoning quality of a 30B+ model.