Best local LLMs for 12 GB of VRAM or unified memory
Measured speeds from builds where the model ran on more than 8 and up to 12 GB of GPU or unified memory. Each number links to its build and source.
8 GB · 12 GB · 16 GB · 24 GB · 32 GB · 48 GB · 64 GB · 96 GB · 128 GB or more
GPUs in this tier: RTX 3060 12GB
Fastest measured result per model
| Model | Plain decode tok/s | With speculative decoding ⚡ | Results |
|---|---|---|---|
| DeepSeek R1 Distill 14B | 70.74 (, RTX 3080 Ti) | – | 4 |
| Llama 3.2 3B | 263.24 (, RTX 3080 Ti) | – | 4 |
| gpt-oss 20B | – | – | 2 |
| Qwen3.8 Flash-Next | 11.5 (UD-IQ3_XXS, experts streamed from RAM/SSD, RTX 5070 12GB) | – | 2 |
| Gemma 4 12B | 38.5 (llama-bench tg128 (quant per result), RTX 3060 12GB) | – | 1 |
| LFM2.5 24B-A2B | – | – | 1 |
| Qwen3.8 27B | – | – | 1 |
Excludes results the source marks pending and results where part of the model sat in system RAM.
All results in this tier
(MTP, DFlash, DSpark and similar draft-and-verify methods give much higher tok/s on predictable text and are not comparable 1:1 with plain decode)
What fits by memory math
Q4_K_M weights (4.8944 bits per weight, per the llama.cpp quantize table) that fit in 12 GB with ~5% headroom, before KV cache: Gemma 4 12B ≈7 GB, Qwen3.5 9B ≈5 GB, Llama 3.1 8B ≈5 GB.
FAQ
Which local LLMs have been measured on 12 GB of VRAM?
DeepSeek R1 Distill 14B, Llama 3.2 3B, gpt-oss 20B, Gemma 4 12B, LFM2.5 24B-A2B, Qwen3.8 27B, Qwen3.8 Flash-Next (15 results).
What fits in 12 GB of VRAM?
By memory math at Q4_K_M (4.8944 bits per weight, llama.cpp), weights only, before KV cache and context: Gemma 4 12B (≈7 GB), Qwen3.5 9B (≈5 GB), Llama 3.1 8B (≈5 GB). Leave a few GB for context.
Are speculative-decoding results comparable?
Not 1:1. MTP, DFlash and similar methods draft tokens that the main model verifies, which can double tok/s on predictable text. This page lists them in a separate column; tick the box to hide them in the full table.
Referral link: if you sign up with Google and add $10, Runpod gives you and RigBase bonus credit (terms). It never affects our data.