Best local LLMs for 8 GB of VRAM or unified memory
Measured speeds from builds where the model ran on up to 8 GB of GPU or unified memory. Each number links to its build and source.
8 GB · 12 GB · 16 GB · 24 GB · 32 GB · 48 GB · 64 GB · 96 GB · 128 GB or more
What fits by memory math
Q4_K_M weights (4.8944 bits per weight, per the llama.cpp quantize table) that fit in 8 GB with ~5% headroom, before KV cache: Gemma 4 12B ≈7 GB, Qwen3.5 9B ≈5 GB, Llama 3.1 8B ≈5 GB.
FAQ
Which local LLMs have been measured on 8 GB of VRAM?
No result in RigBase ran on up to 8 GB of memory yet. The memory-math list on this page shows what should fit.
What fits in 8 GB of VRAM?
By memory math at Q4_K_M (4.8944 bits per weight, llama.cpp), weights only, before KV cache and context: Gemma 4 12B (≈7 GB), Qwen3.5 9B (≈5 GB), Llama 3.1 8B (≈5 GB). Leave a few GB for context.
Are speculative-decoding results comparable?
Not 1:1. MTP, DFlash and similar methods draft tokens that the main model verifies, which can double tok/s on predictable text. This page lists them in a separate column; tick the box to hide them in the full table.
Referral link: if you sign up with Google and add $10, Runpod gives you and RigBase bonus credit (terms). It never affects our data.