Best home rig for Gemma 4 12B
Will it fit my GPU? Best value GPU per euro →
Real, measured generation speeds for Gemma 4 12B on home hardware, fastest first. Different quants/engines aren't directly comparable.
| Build | Quant | Engine | tok/s | Cost | |
|---|---|---|---|---|---|
| RTX 5090: 2026 models (llamabench community) | llama-bench tg128 (quant per result) | llama.cpp | 142 | source | |
| RTX 4090: 2026 models (llamabench community) | llama-bench tg128 (quant per result) | llama.cpp | 98.1 | source | |
| RTX 4090 (llm-speed signed runs, Oct 2026): Qwen3.6 35B-A3B 161 tok/s, Gemma 4 12B QAT 98 tok/s | QAT Q4_0 | llama.cpp b8931 | 97.7 | source | |
| RTX 5070 Ti: 2026 models (llamabench community) | llama-bench tg128 (quant per result) | llama.cpp | 87.6 | source | |
| RTX 3060 12GB: 2026 models (llamabench community) | llama-bench tg128 (quant per result) | llama.cpp | 38.5 | source |
Memory math: how much VRAM / RAM does Gemma 4 12B need?
Gemma 4 12B has 11.91B parameters (source: GGUF param count via llamabench.ai). Weight size = parameters × bits-per-weight ÷ 8, before context (KV cache) and runtime overhead.
| Format | Bits/weight | Weights | Smallest common memory size that holds the weights alone |
|---|---|---|---|
| Q8_0 | 8.50 | 12.7 GB | 16 GB |
| Q6_K | 6.56 | 9.8 GB | 12 GB |
| Q5_K_M | 5.70 | 8.5 GB | 12 GB |
| Q4_K_M | 4.89 | 7.3 GB | 8 GB |
| Q3_K_M | 4.00 | 5.9 GB | 8 GB |
| Q2_K | 3.16 | 4.7 GB | 8 GB |
Bits-per-weight for GGUF quants are llama.cpp's own figures (llama.cpp quantize README). "As measured" rows apply the source's stated bits-per-weight to the parameter count; real files differ slightly because some layers stay at higher precision. Try other combinations in the VRAM calculator.
FAQ
How much VRAM does Gemma 4 12B need?
The weights are about 7.3 GB at Q4_K_M (11.91B params × 4.89 bits ÷ 8, before context/KV cache). Add memory for context (KV cache) and the runtime. The smallest-memory setup with a measured result here, fully in memory, is RTX 3060 12GB (12 GB) at 38.5 tok/s.
What is the fastest measured hardware for Gemma 4 12B?
RTX 5090: 142 tok/s generation (llama.cpp, llama-bench tg128 (quant per result)), from https://llamabench.ai/browse/rtx5090.
Which GPUs have measured results for Gemma 4 12B?
RTX 3060 12GB, RTX 4090, RTX 5070 Ti, RTX 5090.
Can it run on… RTX 3060 12GB · RTX 4090 · RTX 5070 Ti · RTX 5090
Related: what GPU do you need for 2026 models · all builds